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CHAPTER  1:  INTRODUCTION 


1.1  RESEARCH  MOTIVATION 

In  recent  years  the  design  research  community  has  become  increasingly  focused  on 
solving  the  problems  presented  by  uncertain  input  parameters  to  design  methods  and 
algorithms  [Martin  and  Simpson,  2006;  Moeller  and  Beer,  2008;  Schueller  and  Jensen, 
2008;  Lee  and  Chen,  2009].  Design  algorithms,  such  as  design  optimization,  involve 
selecting  (or  determining)  the  physical  parameters  that  define  the  characteristics  of  an 
engineering  system  [Deb,  2001;  Arora,  2004;  Aute  and  Azarm,  2006].  When  dealing 
with  any  physical  parameter  there  is  always  the  possibility  that  the  exact 
dimensions/characteristics  of  that  parameter  cannot  be  known  with  absolute  certainty, 
regardless  of  how  tightly  a  machining  or  fabrication  tolerance  can  be  controlled.  As  a 
result,  uncertainty  is  an  unavoidable  aspect  of  any  engineering  system  and  may  lead  to 
poor  system  performance  and/or  unexpected  system  failure  if  not  adequately  accounted 
for  during  the  design  process  [Jung  and  Lee,  2002;  Li  and  Azarm,  2008].  However, 
understanding  and  mitigating  the  effects  of  uncertainty  on  the  design  of  engineering 
systems  is  a  complex  problem,  especially  when  faced  with  computationally  expensive 
real  world  engineering  design  problems  comprised  of  multiple  engineering  disciplines 
[Du  and  Chen,  2005;  Chiralaksanakul  and  Mahadevan,  2007]. 

Figure  1.1  depicts  just  such  a  system.  Clearly  the  design  of  a  helicopter  is  a 
complex  engineering  challenge  that  involves  designing  many  physically  coupled  systems, 
including  the  fuselage  itself,  the  engines,  and  the  flight  control  computer  that 
communicates  with  the  engines;  just  to  name  a  few.  Furthermore,  designing  these 
systems  could  involve  numerous  distinct  computational  models,  such  as  a  finite  element 
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analysis  (FEA)  model  to  predict  the  behavior  of  the  engine’s  turbine  blades  and  a 
computational  fluid  dynamics  (CFD)  model  to  understand  the  properties  of  the  flow  of  air 
through  the  engine.  Clearly  all  these  systems  and  sub-systems  are  highly  coupled  and 
any  uncertainty  in  one  system  will  invariably  affect  the  design  of  the  other  coupled  sub¬ 
systems  or  disciplines.  There  are  many  important  questions  about  the  effects  of 
uncertainty  on  these  types  of  systems  that  a  designer  would  like  to  answer,  such  as: 
Which  uncertain  parameters  have  the  greatest  effect  on  the  performance  of  the  entire 
system?  Which  sub-systems  are  most  important  to  the  overall  performance  of  the 
system?  If  the  helicopter  needs  to  be  as  light  as  possible  with  as  much  lifting  force  as 
possible,  should  the  engines  be  designed  better  or  should  the  fuselage  itself  be  more 
carefully  addressed?  How  much  uncertainty  in  any  given  parameter  of  the  helicopter’s 
design  should  be  considered  acceptable?  How  best  should  limited  uncertainty  reduction 
and  design  adjustment  resources  be  allocated  during  the  design  process  in  order  to  best 
mitigate  the  potentially  damaging  effects  of  system  uncertainties?  These  are  the  types  of 
questions  the  work  presented  in  this  dissertation  has  attempted  to  address. 


Figure  1.1:  A  Multi-Disciplinary  System 
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1.2  RELEVANT  RELATED  RESEARCH:  AN  OVERVIEW 


1.2.1  Uncertainty  Overview 

Generally  speaking  there  are  two  different  types  of  uncertainty  in  input  parameters: 
irreducible  and  reducible  [Guo  and  Du,  20071.  Irreducible  uncertain  parameters  include 
any  uncertain  factor  or  parameter  that  a  designer  cannot  control,  influence,  prescribe  or 
reduce  further,  even  if  more  information  is  obtained.  A  material  property,  such  as  density, 
is  an  example  of  an  irreducible  uncertain  parameter  due  to  the  natural  variations  in  the 
material.  Reducible  uncertainty  on  the  other  hand  describes  any  uncertain  parameter  that 
a  designer  can  control  the  level  of  uncertainty  in  through  the  specification  of  a  tolerance, 
through  the  purchase  of  higher  quality  equipment,  or  through  the  collection  of  more 
statistical  information.  These  two  types  of  uncertainty  can  be  quantified  in  one  of  three 
ways,  either  through  probability  distributions  [Gunawan  and  Papalambros,  2007;  Noh  et 
al.,  2008;  Youn  and  Wang,  2008],  through  imprecise  probabilities  (e.g.  [Mourelatos  and 
Zhou,  2006;  Zhou  and  Mourelatos,  2008])  or  by  simple  upper  and  lower  bounds  (interval 
uncertainty)  [Du,  2007;  Wu  and  Rao,  2007;  Li  et  al.,  2009a],  At  present,  two  distinct 
strategies  have  been  used  to  manage  input  parameter  uncertainty:  robust  design 
approaches  [Beyer  and  Sendhoff,  2007]  or  Sensitivity  Analysis  (SA)  approaches  [Fiacco, 
1983;  Saltelli  et  al.,  2000;  Helton  and  Davis,  2003],  both  of  which  have  their  strengths 
and  weaknesses. 


3 


1.2.2  Robust  Approaches 


Robust  design  approaches  are  most  appropriate  when  uncertainty  is  irreducible  and 
are  excellent  for  ensuring  the  feasibility  of  an  engineering  system;  thus  preventing  system 
failure  due  to  uncertainty.  It  should  be  noted  that  robust  approaches  cannot  be  counted 
on  to  find  that  a  desired  preexisting  design  of  interest  for  a  system  will  be  robust  (or 
insensitive  to  uncertainty)  and  thus  will  most  likely  suggest  alternate  designs,  which  may 
not  be  something  a  designer  is  interested  in  for  certain  design  problems.  This  is  because 
robust  approaches  traditionally  search  over  a  wide  range  of  potential  designs  in  an  effort 
to  find  the  best  design  that  is  insensitive  to  the  uncertainty  in  the  system  [Gunawan  and 
Azarm,  2005;  Lee,  2006].  For  example,  if  the  designer  of  the  engines  for  the  helicopter 
system  shown  in  Figure  1.1  chose  to  use  a  robust  approach  to  mitigate  the  effects  of 
uncertainty,  the  key  design  parameters  such  as  output  horsepower  and  output  shaft 
diameter  will  likely  be  changed  in  order  to  ensure  insensitivity  to  uncertainty  in  order  to 
accomplish  that  goal.  More  specific  examples  of  robust  methods  are  Robust  Optimization 
(RO)  [Yu  and  Ishii,  1998;  Gu  et.  al,  2006;  Du  and  Choi,  2006;  Apley  et  al.,  2006;  Li  and 
Azarm,  2008]  and  Reliability  Based  Design  Optimization  (RBDO)  [Gunawan  and 
Papalambros,  2006;  Choi  et  al,  2008;  Jung  and  Lee,  2008;  Noh  et  al.,  2008;  Youn  and 
Wang,  2008],  There  are  currently  robust  approaches  specifically  formulated  for 
probabilistic  uncertainty  [e.g.  Youn  and  Wang,  2008],  imprecise  probabilities  [e.g.  Zhou 
and  Mourelatos,  2008]  and  interval  uncertainty  quantification  [e.g.  Li  et  al.,  2007].  These 
approaches  all  treat  input  uncertainties  as  uncontrollable  aspects  of  the  system  that  must 
be  designed  around,  which  is  why  they  are  well-suited  to  systems  where  all  sources  of 
uncertainty  can  be  safely  assumed  to  be  irreducible. 
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However,  in  many  cases  some  uncertainty  sources  may  very  well  be  reducible  and 
the  presence  of  reducible  uncertainty  in  a  system  design  will  undoubtedly  present 
additional  challenges.  More  often  than  not  better  machining  processes,  more  expensive 
equipment  and/or  tighter  tolerances  could  all  reduce  the  uncertainty  in  a  system  and  thus 
result  in  better  system  performance  and  a  reduced  potential  for  failure.  Clearly  all  of 
these  options  for  uncertainty  reduction  can  and  will  increase  the  overall  cost  of  the 
system  design.  For  the  system  in  Figure  1.1,  the  designer  of  the  engines  could  select  the 
best  possible  (and  most  expensive)  thermocouples  available  on  the  market  for  use  in 
monitoring  engine  temperatures  as  precisely  as  possible.  However,  would  the  resulting 
increased  cost  associated  with  those  expensive  thermocouples  be  necessary  to  achieving 
the  overall  goals  and  objectives  of  the  system  as  a  whole,  or  could  those  resources  be 
better  spent  on  some  other  aspect  of  the  design?  Designers  would  obviously  like  to  know 
how  best  to  spend  limited  resources,  thus  achieving  the  best  possible  performance  of  a 
system  under  uncertainty  given  the  lowest  possible  cost,  without  the  fear  that  money  and 
resources  have  been  wasted  on  over  engineering  non-critical  sub-systems  or  parameters. 

For  the  most  part  robust  approaches  in  the  literature  are  not  able  to  assist  designers 
faced  with  these  types  of  decisions  about  reducible  uncertainty.  RBDO  approaches  have 
been  developed  that  allow  for  the  inclusion  of  additional  statistical  information  should  it 
become  available  as  a  means  for  considering  reducible  uncertainty  [Gunawan  and 
Papalambros,  2006;  Youn  and  Wang,  2008].  Other  approaches  combine  uncertainty 
reduction  mechanisms  with  RBDO  techniques,  either  through  sequentially  performing 
reliability-based  design  optimization  followed  by  uncertainty  reduction  [Qu  et  al.,  2003], 
or  through  simultaneously  considering  reliability  and  uncertainty  reduction  mechanisms 
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during  the  design  process  [Kale  and  Haftka,  2008].  There  are  also  approaches  presented 
in  the  literature  that  suggest  treating  the  variance  in  an  uncertain  parameter  as  a  factor  to 
control  within  an  RBDO  algorithm  provided  a  designer  has  a  means  for  producing  lower 
variance  in  input  parameters  [Benanzer  et  al.,  2009].  However,  the  above  approaches  all 
require  probabilistic  information  or  imprecise  probabilities  for  uncertainty  quantification, 
which  may  be  unavailable,  undesirable  or  invalid  when  statistical  information  is  not 
available  or  too  expensive  to  obtain  early  in  a  design  process. 

1.2.3  Sensitivity  Analysis  Approaches 

Sensitivity  analysis  (SA)  based  approaches  are  often  better  suited  to  finding 
opportunities  for  reducing  any  uncertainty  levels  in  an  engineering  system  design  that  are 
known  to  be  reducible  [Greenland,  2001;  Acar  et  al.,  2007;  Wu  and  Rao,  2007;  Li  et  al., 
2009a],  This  is  because  sensitivity  analysis  simply  provides  a  designer  with  information 
relating  output  variation  to  input  uncertainty.  That  information  can  then  be  used  to  drive 
uncertainty  reduction  decisions  if  desired.  SA  based  approaches  are  also  very  useful 
when  a  designer  is  interested  in  the  effects  of  uncertainty  on  a  predetermined  design  of 
interest  [Iman  and  Helton,  1988;  Sobieszczanski-Sobieski,  1990].  If  SA  is  used  to 
analyze  the  design  of  the  engine  in  Figure  1.1,  the  nominal  values  of  the  output  power 
and  shaft  diameter  are  more  likely  to  remain  largely  unchanged  for  a  preexisting  design 
of  interest  when  using  SA.  This  is  because  SA  based  approaches  can  provide  a  designer 
with  information  about  the  system  that  can  be  used  to  mitigate  uncertainty  in  another 
manner.  SA  approaches  can  be  classified  as  either  local  or  global.  Local  SA  approaches 
relate  the  variation  in  the  outputs  of  a  system  to  small  uncertainty  in  the  system  inputs 
[Hamby,  1994;  Frey  and  Patil,  2002;  Kern  et  al.,  2003],  which  is  only  valid  for  limited 
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levels  of  uncertainty.  On  the  contrary,  global  SA  approaches  [Saltelli  et  al.,  2008] 
account  for  the  entire  range  of  input  uncertainty  and  determines  the  effects  of  input 
uncertainty  on  system  outputs.  Most  of  the  SA  approaches  reported  in  the  literature  focus 
on  probabilistic  uncertainty  quantification  [e.g.  Acar  et  al.,  2007;  Castillo  et  al.,  2008; 
Du,  2008],  in  part  because  the  acquisition  of  more  data  can  often  be  used  to  reduce 
uncertainty  in  a  probabilistic  sense.  However,  some  attempts  have  been  made  to  apply 
SA  based  approaches  to  systems  possessing  interval  uncertainty  [Wu  and  Rao,  2007;  Li 
et  al.,  2009a;  Li  et  al.,  2009b],  Sensitivity  analysis  techniques  are  ideally  suited  to  relate 
system  performance  variation  (and/or  system  failure)  to  input  uncertainty.  However, 
there  are  not  many  SA  based  approaches  that  can  then  also  determine  the  design  changes 
and/or  uncertainty  reductions  required  to  improve  the  performance  (or  eliminate  the 
possibility  of  failure)  of  a  design  under  uncertainty,  with  Li’s  work  being  the  only  major 
exception  [Li  et  al.,  2009a,  Li  et  al.,  2009b]. 

1.2.4  Multi-Disciplinary  Approaches 

Much  of  the  work  done  in  this  area  of  design  under  uncertainty  has  focused  on 
single  disciplinary  engineering  systems,  including  both  robust  approaches  [Crespo  et  al., 
2008]  and  SA  approaches  [Helton  and  Davis,  2003].  Furthermore,  SA  approaches 
capable  of  handling  multiple  system  outputs  are  relatively  rare  in  the  literature  [Barron 
and  Schmidt,  1988;  Zhang,  2003;  Li  et  al.,  2009a].  However,  engineering  systems  of 
interest  will  possess  multiple  design  goals  (or  objectives)  and  will  most  certainly  be 
comprised  of  many  different  disciplines.  As  discussed  above  in  the  helicopter  engine 
example,  those  multiple  disciplines  could  be  in  the  form  of  coupled  computational 
models,  such  as  an  FEA  model  for  a  structure  and  a  CFD  model  for  analyzing  the  drag 
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forces  on  the  structure.  Alternately,  the  multiple  disciplines  could  be  in  the  form  of 
geometrically  coupled  but  distinct  sub-systems  of  a  greater  engineering  system,  such  as 
the  motor  and  battery  modules  that  comprise  the  design  of  a  power  tool  which  are  to  be 
designed  by  different  teams  possibly  in  different  geographic  locations. 

1.3  RESEARCH  FOCUS  AND  KEY  ASSUMPTIONS 

The  work  presented  in  this  dissertation  has  sought  to  address  many  of  the 
shortcomings  of  the  current  research  in  the  area  of  design  under  uncertainty  as  described 
in  the  previous  section.  The  overall  goal  of  this  work  has  been  to  provide  designers  with 
sensitivity  analysis  based  algorithms  for  understanding  and  then  optimally  mitigating  the 
effects  of  input  uncertainty  on  single  and  multi-disciplinary  engineering  systems  that  are 
as  computationally  efficient  as  possible.  In  order  to  achieve  this  goal  the  research 
presented  in  this  dissertation  has  focused  on  the  following  five  key  areas  and  their 
corresponding  assumptions. 

Sensitivity  Analysis  (SA):  As  described  previously,  SA  based  approaches  are 
effective  tools  for  better  understanding  the  effects  of  uncertainty  on  a  preexisting  design. 
However,  most  SA  based  approaches  currently  in  the  literature  are  simply  tools  for 
gaining  greater  insight  into  the  effects  of  uncertainty  on  an  engineering  system.  As  a 
result,  it  is  then  up  to  the  designer  how  best  to  proceed  using  any  newly  gained  insight. 
Much  of  the  work  presented  in  this  dissertation  combines  SA  with  multi-objective 
optimization  techniques  to  provide  a  designer  both  with  greater  insight  into  the  effects  of 
uncertainty  and  more  importantly  with  options  for  how  best  to  adjust  any  available 
degrees  of  freedom  in  the  system  to  mitigate  the  effects  of  uncertainty.  As  such,  much  of 
the  research  presented  in  this  dissertation  has  assumed  that  a  preexisting  design  for  an 
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engineering  system  of  interest  exists  and  is  known  to  possess  uncertain  input  parameters. 
Figure  1.2  depicts  the  general  capability  of  SA,  which  is  to  relate  output  variations  to 
input  uncertainties. 


Figure  1.2:  Sensitivity  Analysis 

Engineering  Feasibility:  Perhaps  the  most  important  effect  of  uncertainty  that  needs 
to  be  mitigated  in  any  engineering  design  effort  is  the  potential  for  system  failure  due  to 
uncertainty.  This  concept  of  engineering  failure  when  placed  in  the  context  of 
engineering  optimization  is  also  known  as  infeasibility.  Previous  work  in  the  area  of 
sensitivity  analysis  has  focused  on  reducing  variations  in  system  outputs  but  there  have 
only  been  limited  efforts  to  address  system  failure  specifically.  This  work  has  focused 
specifically  on  this  key  area  in  engineering  design  and  how  sensitivity  analysis  can  be 
used  to  ensure  engineering  feasibility,  in  both  single  and  multi-disciplinary  systems. 

Interval  Uncertainty:  While  probability  distributions  are  not  always  available  for 
uncertain  parameters  (and  obtaining  such  information  is  often  quite  expensive,  if  even 
possible),  interval  uncertainty  levels  are  often  much  easier  to  determine  and/or  estimate. 
For  this  reason  the  sensitivity  analysis  based  work  presented  in  this  dissertation  has 
focused  exclusively  on  interval  uncertainty  in  an  effort  to  be  as  general  (and  easy  to 
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implement)  as  possible.  Note  that  this  work  has  focused  on  system  input  parameter 
uncertainty  and  does  not  specifically  address  any  model  uncertainty  potentially  associated 
with  the  analysis  models  of  the  system  being  analyzed. 

Reducible  Uncertainty:  As  previously  discussed,  the  presence  of  reducible  sources 
of  input  uncertainty  presents  a  distinct  challenge  to  engineering  system  designers.  When 
uncertainty  is  potentially  reducible  a  designer  is  forced  to  make  decisions  about  the 
improvement  in  performance  that  can  be  obtained  through  uncertainty  reduction  while 
simultaneously  taking  into  account  the  increased  cost  of  further  reducing  uncertainty. 
The  research  presented  in  this  dissertation  attempts  to  provide  system  designers  with 
multiple  approaches  for  addressing  this  challenge  in  a  systematic  and  automated  fashion, 
thus  making  the  process  of  design  under  reducible  uncertainty  a  simpler  task. 

Algorithm  Efficiency:  For  any  design  algorithm  to  be  useful  it  must  be  very 
efficient  as  current  computational  models  for  engineering  systems  can  take  many  hours  to 
execute.  In  order  to  address  this  very  real  issue,  all  the  work  presented  in  this  dissertation 
has  attempted  to  increase  the  computational  efficiency  of  the  approaches  developed 
through  the  use  of  surrogate  approximation  models  wherever  possible.  Surrogate 
modeling,  also  widely  known  as  meta-modeling,  describes  a  set  of  techniques  for  using 
limited  actual  analysis  data  to  approximate  a  complex  computational  model  [e.g.  Shan 
and  Wang,  2008].  The  work  presented  in  this  dissertation  have  used  analysis  data 
generated  early  in  design  procedures  to  build  surrogate  models  that  are  then  used  during 
later  steps  to  greatly  reduce  computational  effort  through  the  intelligent  re-use  of  already 
obtained  system  information. 
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1.4  RESEARCH  THRUSTS 


To  accomplish  the  overall  goals  of  this  dissertation  research,  four  distinct  research 
thrusts  were  pursued.  Each  thrust  built  on  the  work  of  the  previous  thrust  by  relaxing 
some  previous  assumptions  and/or  addressing  issues  that  were  ignored  or  avoided  in 
previous  thrusts.  Only  interval  uncertainty  was  considered  throughout  the  research  and 
the  interval  uncertainty  levels  associated  with  each  design  under  consideration  were 
assumed  to  be  reducible  to  some  extent.  Each  of  four  research  thrusts  performed  will  be 
described  briefly  in  the  following  paragraphs  and  then  in  detail  later  in  this  dissertation. 

1.4.1  Thrust  1:  Multi-Disciplinary  Multi-Output  Sensitivity  Analysis 

The  first  step  in  this  dissertation  research  was  to  extend  the  work  of  Li  et  al. 
[2009a]  to  multi-disciplinary  systems.  As  such,  a  sensitivity  analysis  based  uncertainty 
reduction  approach,  called  Multi-disciplinary  Multi-Output  Sensitivity  Analysis 
(MIMOSA)  was  developed.  This  approach  was  developed  for  multi-disciplinary 
engineering  systems  decomposed  into  multiple  sub-systems,  where  each  sub-system 
analysis  model  has  multiple  uncertain  inputs  and  multiple  outputs.  MIMOSA  can 
determine:  i)  the  sensitivity  of  system  and  sub-system  outputs  to  input  uncertainties  at 
both  system  and  sub-system  levels,  ii)  the  sensitivity  of  the  system  outputs  to  the 
variations  from  sub-system  outputs,  and  iii)  the  optimal  “investment”  required  to  reduce 
uncertainty  in  inputs  in  order  to  obtain  a  maximum  reduction  in  output  variations  at  both 
the  system  and  sub-system  levels.  A  numerical  and  an  engineering  example  with  two  and 
three  sub-systems,  respectively,  were  used  to  demonstrate  the  applicability  of  the 
MIMOSA  approach.  A  portion  of  this  research  thrust  was  presented  in  Li  et  al.,  [2009b] 
and  the  details  of  this  new  approach  will  be  presented  in  Chapter  3  of  this  dissertation. 
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The  objective  of  this  research  thrust  was  to  develop  an  approach  for  understanding 
and  optimally  mitigating  the  effects  of  reducible  interval  input  uncertainty  on  multi¬ 
disciplinary,  multi-output  engineering  systems. 

1.4.2  Thrust  2:  Multi-Disciplinary  Combined  Sensitivity  Analysis 

The  goal  of  the  second  research  thrust  was  to  develop  a  sensitivity  analysis  method 
for  determining  where  the  appropriate  uncertainty  reduction  opportunities  exist  in  a 
preexisting  multi-disciplinary  system  design  in  an  effort  to  ensure  the  feasibility  of  all 
included  sub-system  designs.  This  approach  builds  on  the  work  of  the  previous  thrust, 
which  did  not  consider  engineering  feasibility.  Using  the  MIMOSA  approach  as  a 
starting  point,  the  Multi-disciplinary  COmbined  Sensitivity  Analysis  (MICOSA) 
approach  was  formulated  to  achieve  both  minimal  output  function  variation  at  the  system 
and  sub-system  levels  (like  MIMOSA),  while  simultaneously  ensuring  that  both  the 
overall  system  and  all  included  sub-systems  will  be  feasible  in  the  presence  of  any 
retained  uncertainty.  MICOSA  combines  output  function  variation  reduction  with  the 
equally  important  goal  of  preventing  the  possibility  of  engineering  failure  due  to  input 
uncertainty.  This  approach  was  applied  to  a  notional  multi-disciplinary  unmanned 
underwater  vehicle  model  to  demonstrate  its  capabilities.  The  details  of  this  approach  are 
presented  in  Chapter  4. 

The  objective  of  the  second  research  thrust  was  to  extend  the  MIMOSA  approach  to 
also  consider  engineering  feasibility,  thus  making  it  possible  to  use  uncertainty  reduction 
mechanisms  as  a  means  for  both  optimally  reducing  output  function  variations  while 
simultaneously  ensuring  the  feasibility  of  multi-disciplinary  engineering  systems. 
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1.4.3  Thrust  3:  Design  Improvement  by  Sensitivity  Analysis 


Uncertainty  in  the  input  parameters  to  an  engineering  system  may  not  only  degrade 
the  system’s  performance,  but  may  also  cause  failure  or  infeasibility,  as  was  addressed  in 
the  second  research  thrust.  However,  MICOSA  only  uses  uncertainty  reduction 
mechanism  as  a  means  for  achieving  feasibility  under  uncertainty,  which  may  not  always 
be  sufficient.  The  third  research  thrust  addressed  this  limitation  of  MICOSA  and  focused 
on  the  development  of  a  new  sensitivity  analysis  based  approach  called  Design 
Improvement  by  Sensitivity  Analysis  (DISA).  DISA  analyzes  the  interval  uncertainty  of 
input  parameters  and,  using  multi-objective  optimization,  determines  an  optimal 
combination  of  design  improvements  that  will  ensure  a  minimal  variation  in  the  objective 
functions  of  the  system  while  also  ensuring  feasibility.  The  approach  provides  a  designer 
with  options  for  both  uncertainty  reduction  and,  more  importantly,  slight  design 
adjustments  (which  will  be  defined  in  detail  later  in  this  dissertation).  A  two  stage 
sequential  framework  is  used  that  can  employ  either  the  original  analysis  functions,  or 
meta-model  approximations,  to  greatly  increase  the  computational  efficiency  of  the 
approach.  This  new  approach  has  been  applied  to  two  engineering  examples  of  varying 
difficulty  to  demonstrate  its  applicability  and  effectiveness.  The  results  produced  by 
these  examples  show  the  ability  of  the  approach  to  ensure  the  feasibility  of  a  preexisting 
design  under  interval  uncertainty  by  effectively  adjusting  available  degrees  of  freedom  in 
the  system  without  the  need  to  completely  redesign  the  system.  A  portion  of  this  work 
was  presented  in  Hamel  et  al.  [2010]  and  the  details  of  this  research  thrust  are  presented 
in  Chapter  5  of  this  dissertation. 
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The  objective  of  the  third  research  thrust  was  to  develop  a  sensitivity  analysis 
based  approach  for  optimally  mitigating  the  effects  of  reducible  interval  input 
uncertainty  on  a  preexisting  candidate  design  that  is  capable  of  both  engineering 
feasibility  and  minimal  objective  function  variations  at  a  minimal  require  cost  through 
the  use  of  uncertainty  reduction  mechanisms  and  small  design  adjustments. 

1.4.4  Thrust  4:  Reducible  Uncertain  Interval  Design 

Optimization  under  uncertainty  can  be  a  difficult  and  computationally  expensive 
problem  driven  by  the  need  to  consider  the  degrading  effects  of  system  variations,  as 
discussed  earlier  in  this  chapter.  Sources  of  uncertainty  that  may  be  reducible  in  some 
fashion  present  a  particular  challenge  because  designers  must  determine  how  much 
uncertainty  to  accept  in  the  final  design.  As  previously  discussed,  many  of  the  existing 
approaches  for  design  under  input  uncertainty,  such  as  MIMOSA  and  DISA  for  example, 
require  potentially  unavailable  or  unknown  information  about  the  uncertainty  in  a 
system’s  input  parameters;  such  as  probability  distributions,  nominal  values  or  uncertain 
intervals.  These  requirements  may  force  designers  into  arbitrary  or  even  erroneous 
assumptions  about  a  system’s  input  uncertainty  when  attempting  to  estimate  nominal 
values  and/or  uncertain  intervals  for  example.  These  types  of  assumptions  can  be 
especially  degrading  during  the  early  stages  in  a  design  process  when  limited  system 
information  is  available.  In  an  effort  to  address  these  challenges  a  new  SA  inspired 
design  approach  was  developed  that  can  produce  optimal  solutions  in  the  form  of  upper 
and  lower  bounds  (which  specify  uncertain  intervals)  for  all  input  parameters  to  a  system 
that  possess  reducible  uncertainty.  These  solutions  provide  minimal  variation  in  system 
objectives  for  a  maximum  allowed  level  of  input  uncertainty  in  a  multi-objective  sense 
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and  furthermore  guarantee  as  close  to  deterministic  Pareto  optimal  performance  as 
possible  with  respect  to  the  uncertain  parameters.  The  function  calls  required  by  this 
approach  are  dramatically  reduced  through  the  use  of  a  kriging  meta-model  assisted 
multi-objective  optimization  technique  performed  in  two  stages.  The  capabilities  of  the 
approach  are  demonstrated  through  three  example  problems  of  varying  complexity.  This 
final  research  thrust  seeks  to  relax  many  of  the  limiting  assumptions  of  the  MIMOSA, 
MICOSA  and  DISA  approaches  in  an  effort  to  provide  designers  with  a  general  approach 
for  the  design  of  engineering  system  under  reducible  interval  uncertainty.  A  portion  of 
this  work  was  presented  in  Hamel  and  Azarm  [2010]  and  the  details  of  this  final  research 
thrust  are  presented  in  Chapter  6  of  this  dissertation. 

The  objective  of  this  fourth  and  final  research  thrust  was  to  develop  an  sensitivity 
analysis  inspired  approach  for  the  multi-objective  optimal  design  of  engineering  systems 
under  reducible  interval  input  uncertain  that  is  capable  of  determining  the  optimal  upper 
and  lower  bounds  for  sources  of  reducible  input  uncertainty  to  an  engineering  system 
with  as  few  required  a  priori  assumptions  as  possible. 

1.5  ORGANIZATION  OF  DISSERTATION 

The  research  thrusts  introduced  in  the  previous  section  represent  the  major 
contributions  of  the  work  presented  in  this  dissertation.  As  previously  discussed  the  four 
research  thrusts  are  interrelated  and  each  thrust  successively  builds  up  on  the  work  of  the 
previous  thrusts.  Figure  1.3  below  depicts  the  relationships  between  the  four  thrusts 
graphically. 
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Figure  1.3:  Organization  of  Research  Thrusts 

As  can  be  seen  in  Figure  1.3,  the  new  approach  presented  in  each  thrust  is  either 
more  capable  and/or  requires  less  a  priori  information  from  the  designer  to  produce 
similar  results.  For  example,  in  the  first  research  thrust  a  new  approach  for  using 
uncertainty  reduction  to  control  the  output  variations  in  a  multi-disciplinary  system  is 
presented.  Then,  the  second  thrust  extends  the  approach  presented  in  the  first  to  also 
consider  engineering  feasibility,  which  was  not  considered  previously.  This  progression 
resulted  in  a  set  of  four  new  SA  based  approaches  for  the  design  of  engineering  systems 
under  reducible  interval  uncertainty  where  each  new  approach  presented  addresses  the 
weaknesses  and/or  limitations  of  the  previously  presented  approaches. 


The  remainder  of  this  dissertation  will  be  organized  as  follows.  Chapter  2  contains 
some  relevant  background  definitions  and  terminology  that  will  be  used  throughout  this 
dissertation.  Chapters  3  through  6  cover  the  details  of  the  four  main  research  thrusts. 
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Chapter  7  contains  the  relevant  conclusions  developed  in  each  of  the  four  research 


thrusts,  an  overview  of  the  major  contributions  of  this  work  and  some  potential  directions 
for  future  research. 
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CHAPTER  2:  BACKGROUND  AND  TERMINOLOGY 


2.1  INTRODUCTION 

In  this  chapter  some  general  background  information,  definitions  and  terminology 
will  be  presented.  These  terms,  equations  and  figures  will  be  referred  to  throughout  the 
dissertation. 


2.2  MULTI-OBJECTIVE  OPTIMIZATION 


Multi-objective  optimization  involves  solving  the  problem  shown  in  Eqn.  (2.1), 
where  /  describes  a  set  of  objective  functions  for  a  system  which  depend  on  a  vector  of 
design  variables  x  and  a  vector  of  parameters  p,  and  limited  by  a  set  of  constraint 
functions  g.  If  any  of  the  constraint  functions  g  in  Eqn.  (2.1)  has  a  value  greater  than  0, 
the  system  is  said  to  be  infeasible.  Typically  Eqn.  (2.1)  would  also  contain  a  set  of 
equality  constraint  functions  as  well,  but  these  have  been  omitted  in  this  work  as  most 
equality  constraints  will  necessarily  be  violated  under  uncertainty. 

min  f,(x,p)  i  =  1,...,  / 

x 


s.t. 

gj(x,p)<  0  j  = 


(2.1) 


Traditionally  the  vector  x  describes  factors  that  a  designer  can  control  while  p  is  a 
vector  of  uncontrolled  inputs.  However,  if  the  elements  of  both  x  and p  are  either  known, 
in  the  form  of  a  candidate  design  (as  required  by  most  SA  procedures),  or  potentially 
controllable  in  some  sense  (because  the  parameter  or  its  upper  and/or  lower  bounds  are 
selectable),  then  the  system  can  be  thought  of  as  depending  simply  on  a  single  vector  of 
input  parameters p  =  {x  ,p]  (assuming  x  and p  are  row  vectors)  as  shown  in  Eqn.  (2.2). 
In  most  cases  this  input  vector  is  bounded  by  extreme  upper  and  lower  bounds  as  shown. 
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From  this  point  forward  there  will  be  no  distinction  between  x  and  p.  All  inputs  to  an 
analysis  model  will  simply  be  referred  to  as  parameters  p  as  shown  in  Eqn.  (2.2). 

min  f.(p)  i  =  l,...,I 

P 

s.t. 


gj(p)^  0  j  = 

Pw^P^  Pu„ 

where  p  =  {pl,...,pk,...,pK} 


(2.2) 


From  this  point  forward,  a  set  of  values  for  the  vector  p  will  be  referred  to  as  a 
design  and  evaluating  the  values  for  the  actual  functions /  and  g  for  a  single  design  p  will 
be  referred  to  as  a  function  call.  Multi-objective  optimization  can  be  accomplish  with 
many  different  techniques  [Miettinen,  1999;  Deb,  2001],  but  the  work  presented 
throughout  this  dissertation  uses  Multi-Objective  Genetic  Algorithms  (MOGA)  in  order 
to  accommodate  mixed  continuous-discrete  input  parameters  and  discontinuous  output 
functions  [Deb,  2001]. 


2.3  MULTI-DISCIPLINARY  DESIGN  OPTIMIZATION 

When  the  analysis  model  or  simulation  model  for  an  engineering  system  contains 
more  than  one  coupled  sub-system  analysis  models  then  the  system  is  said  to  be  multi¬ 
disciplinary  and  optimizing  the  design  of  such  a  system  is  called  Multi-disciplinary 
Design  Optimization  (MDO).  An  MDO  problem  can  be  solved  in  an  all-at-once  (i.e., 
with  all  coupled  sub-systems  considered  together  in  a  single  system)  or  a  decomposed 
(i.e.,  with  decoupled  sub-systems)  fashion.  Some  examples  of  MDO  decomposition 
techniques  include  Concurrent  Subspace  Optimization  (CSSO),  Collaborative 
Optimization  (CO)  [Yi  et  al.,  2007]  and  Analytical  Target  Cascading  (ATC)  [Kokkolaras 
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et  al.,  2006],  all  of  which  involve  nesting  sub-system  optimizations  within  system  level 
optimizers.  The  decomposed  solution  is  more  useful  in  that  it  allows  a  designer  to 
consider  and  evaluate  the  performance  of  each  sub-system  individually.  Figure  2. 1  below 
shows  the  relationship  between  the  all-at-once  and  decomposed  MDO  formulations  using 
the  Collaborative  Optimization  (CO)  technique.  The  major  difference  between  the  two 
approaches  in  the  figure  is  the  addition  of  a  new  type  of  variables  for  the  decomposed 
approach,  target  variables  t,  at  the  system  level,  which  serves  as  a  target  for  the  coupling 
variables  y  in  each  sub-system  at  the  sub-system  level.  An  additional  set  of  constraints 
are  also  added  in  the  decomposed  approach  to  ensure  that  at  the  sub-system  level  each  of 
the  coupling  variables  y  is  equal  to  its  target  variable  value.  This  technique  was  extended 
to  multiple  system  objectives  by  Aute  and  Azarm  [2006],  and  their  approach  is  used 
throughout  this  dissertation  when  an  MDO  framework  is  required.  Note  that  the 
subscript  sh  in  Figure  2. 1  denotes  input  at  the  system  level  shared  by  more  than  one  of 
the  sub-systems. 


SSO 


All- At -Once 


Xl,Pl 

System  Level  Sub-System  Level 


Figure  2.1:  Decomposition  of  a  Multi-Disciplinary  System 
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2.4  CANDIDATE  DESIGN 


In  order  to  perform  sensitivity  analysis  the  assumption  is  that  the  variables  p  in 
Eqn.  (2.2)  are  already  given  in  the  form  of  a  predetermined  design.  Thus  hereafter  in  this 
dissertation  the  vector  po  is  used  to  denote  the  nominal  values  for  all  inputs  into  a  system 
for  which  a  solution,  or  design,  already  exists.  Furthermore  this  design  po  will  be  called  a 
candidate  design.  As  a  result,  the  nominal  objective  and  constraint  functions  values  for  a 
candidate  design  may  be  thought  of  simply  as  functions  of  the  nominal  values  po,  which 
is  a  vector  that  contains  both  the  predetermined  design  variable  values  and  the  parameter 
values  from  the  original  engineering  design  problem  in  Eqn.  (2.1).  The  work  presented  in 
this  dissertation  focuses  on  how  changes  to  the  parameters  contained  in  the  vector  po, 
both  in  terms  of  nominal  values  and  uncertainty  level,  affect  the  performance  of  the 
design  described  by  the  functions  /o  and  go. 

2.5  INPUT  PARAMETER  UNCERTAINTY 

Interval  uncertainty  is  considered  in  this  work  due  to  the  ability  of  a  designer  to 
define  uncertainty  given  limited  information  about  parameters  during  the  early  stages  of  a 
design  process.  As  stated  earlier  in  this  chapter,  throughout  this  dissertation  any  input  to  a 
system  analysis  model  will  be  considered  a  parameter.  The  interval  uncertainty  possessed 
by  the  kxh  element  of  the  vector  of  input  parameters  p  is  defined  as  A/;*.  For  notational 
simplicity  it  is  assumed  that  the  Apt  associated  with  any  parameter  is  symmetric.  In  other 
words,  the  difference  between  the  nominal  parameter  values  po  and  the  upper  bounds  on 
those  parameters  pu  is  the  same  as  the  difference  between  the  nominal  values  and  the 
lower  bounds  pL,  or  for  the  kth  uncertain  interval  Ap/,  =  I  p0tk  -  pUtk  I  =  I  p0,k  -  pi„k  I.  This 
assumption  is  not  necessary  though  and  asymmetric  uncertainty  levels  can  be  dealt  with 
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easily  by  considering  two  A p  values  for  each  uncertain  parameter  po,k ■  In  two  dimensions 
the  uncertainty  quantified  this  way  can  be  depicted  as  a  rectangle  in  the  parameter  space 
for  a  problem,  as  shown  in  Figure  2.2.  The  parameter  space  can  be  thought  of  as  the 
entire  range  of  all  possible  parameter  combinations  for  a  given  system.  The  uncertainty 
associated  with  the  candidate  design  p0  is  a  subset  of  the  parameter  space,  bounded  by  the 
interval  [po  -  &p.  pu+  \p]  (as  originally  defined  by  Moore  [1966]). 

Pi 


Figure  2.2:  Parameter  Space  and  Parameter  Uncertainty 

2.6  REDUCIBLE  PARAMETER  UNCERTAINTY 

If  the  uncertainty  in  a  set  of  parameters  is  reducible  then,  as  proposed  by  Li  et  al. 
[2009a],  reduction  in  that  uncertainty  can  be  quantified  using  a  vector  of  uncertainty 
reduction  scaling  factors  a  =  [ct],...,  ak,...,af(}.  For  each  reducible  input  uncertainty  level 
A pk  to  a  system,  a  scaling  factor  ak  is  assigned.  Each  of  the  K  elements  of  the  vector  a  is 
a  number  between  0  and  1 .  The  values  of  a  and  A p  work  together  to  reduce  the  original 
uncertain  intervals  oip  as  shown  in  Figure  2.3.  The  new  (smaller)  uncertain  region  in  the 
parameter  space  is  calculated  as  the  Hadamard  product  of  the  uncertainty  reduction 
vector  and  the  original  vector  of  parameter  uncertainties:  a  °  Ap  =  {a.\Ap\,..., 
akApk aKApK).  If  the  uncertainty  level  for  a  specific  parameter  cannot  be  reduced 
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completely,  as  is  usually  the  case,  a  lower  bound  can  be  placed  on  the  corresponding  «* 
value.  Since  the  values  for  a  can  be  any  value  between  0  and  1,  the  uncertainty  in  any 
parameter  p  can  be  reduced  to  any  subset  of  values  on  the  original  uncertain  interval  [po  - 
Ap,  po  +  \p].  For  instance,  if  uncertainty  in  p\  is  completely  irreducible  then  a\  should 
be  limited  to  always  be  equal  to  1;  while  if  the  uncertainty  in  p\  can  only  be  reduced  by 
50%,  then  a\  is  limited  to  a  minimum  value  of  0.5.  Additionally,  if  any  of  the  available 
uncertainty  reduction  options  for  the  parameter  p*  are  discrete  choices,  then  the  available 
a.k  options  can  be  limited  to  corresponding  discrete  values  between  0  and  1.  It  should  be 
noted  that  the  work  presented  in  this  dissertation  assumes  that  uncertain  intervals  are 
symmetric  about  the  nominal  values  po  and  thus  only  one  element  of  the  uncertainty 
reduction  vector  a  is  required  for  each  reducible  input  parameter  of  interest.  In  the  case 
of  an  asymmetric  uncertainty  about  the  nominal,  an  additional  element  should  be  added 
to  the  a  vector  (increasing  the  size  of  the  vector)  for  each  input  parameter  with 
asymmetric  uncertainty.  When  using  these  additional  elements,  one  element  of  the  a 
vector  controls  the  reduction  in  the  upper  bound  of  the  uncertain  interval  for  a  parameter 
while  the  additional  element  controls  the  reduction  in  the  lower  bound.  This  modification 
increases  the  dimensionality  of  the  a  vector,  but  also  gives  a  designer  greater  control  over 
uncertainty  reduction  decisions  if  necessary. 


|p0-  a  °Ap ,  p0  +  a  °Ap] 
Pi 

«2Ap2 


Pi 


Figure  2.3:  Parameter  Uncertainty  Reduction 
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If  parameter  uncertainty  is  reduced  by  decreasing  the  values  for  one  or  more 
elements  in  a,  the  range  of  output  function  variation  will  also  inevitably  be  changed. 
This  propagation  of  uncertainty  from  input  parameters  to  output  function  values  can  be 
represented  by  a  set  of  resulting  uncertain  objective  and/or  constraint  function  values.  As 
discussed  above,  uncertainty  propagation  is  a  function  of  the  nominal  parameter  values 
po ,  the  original  uncertainty  intervals  Ap,  and  the  values  of  the  uncertainty  reduction 
vector  a.  Figure  2.4  depicts  the  impact  that  different  uncertainty  reduction  values 
(denoted  by  decreasing  a  values)  could  have  on  output  objective  function  variations  when 
nominal  parameter  values  p0  remain  unchanged. 


Figure  2.4:  Parameter  Uncertainty  Mapping  for  Various  aValues 


2.7  UNCERTAINTY  QUANTIFICATION  METRICS 

In  order  to  effectively  manage  input  uncertainty  in  an  effort  to  improve  output 
function  performance  (i.e.,  to  reduce  the  variation  in  the  outputs),  it  is  necessary  to 
measure  and  quantify  objective  and  constraint  function  variations  as  a  function  of  any 
uncertainty  in  the  system’s  input  parameters.  In  order  to  do  this,  Li  et  al.’s  work  [2009a] 
is  extended  in  this  dissertation  to  consider  either  objective  functions  /,  constraint 
functions  g,  or  both.  If  the  variations  in  the  objective  functions  are  of  interest,  the  metric 
used  is  call  /?/,  while  the  measure  of  constraint  functions  variations  under  uncertainty  will 
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be  called  Rg.  If  both  objectives  and  constraints  are  considered  simultaneously  their 
variations  will  be  measured  with  a  metric  called  Rc,  while  if  the  nature  of  the  output 
functions  (objectives  or  constraints)  is  not  specified,  a  general  metric  called  R  will  be 
used.  Figure  2.5  below  depicts  two  different  notional  ft/ values  for  a  notional  bi-objective 
system.  Similar  figures  could  be  drawn  for  ft?,  ftc  and/or  ft  as  needed. 


Figure  2.5:  Graphical  Depiction  of  ft/ 

Note  in  Figure  2.5  that  ft/  (or  any  output  function  variation  for  that  matter)  will 
always  be  a  function  of  any  changes  to  the  nominal  input  parameters  p0  and  their 
associated  uncertainty.  For  example,  in  the  case  of  uncertainty  reduction,  ft/  will  be  a 
function  of  the  elements  of  the  uncertainty  reduction  vector  a  which  describes  the  level  of 
uncertainty  reduction  associated  with  a  system  design,  as  discussed  in  the  previous 
subsection.  This  is  depicted  in  Figure  2.5  by  the  light  colored  objective  variation  region 
having  a  smaller  ft/ than  the  larger  darker  colored  region,  which  has  a  higher  associated  ft/ 
value.  In  the  subsequent  discussions  of  each  of  the  research  thrusts  the  specific  variation 
metrics  used  (i.e.  ft/,  RK  and/or  Rc)  will  be  defined  more  specifically  as  required. 

2.8  INVESTMENT 

As  a  measure  of  defining  the  cost  of  reducing  uncertainty  in  input  parameters  the 
concept  of  Investment  was  developed  by  Li  et  al.  [2009a].  The  Investment  metric  is 
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essentially  a  notional  function  that  correlates  the  amount  of  uncertainty  associated  with  a 
candidate  design  to  the  “cost”  required  to  produce  that  uncertainty  level.  The  general 
concept  of  the  Investment  metric  is  presented  for  a  two  parameter  system  in  Figure  2.6, 
which  shows  that  a  higher  reduction  in  uncertainty  from  the  nominal  will  require  a 
greater  Investment  level. 


Low  Investment  High  Investment 

Figure  2.6:  General  Investment  Concept 

Various  formulations  of  Li  et  al.’s  [2009a]  Investment  metric  are  used  throughout 
this  work  and  the  specific  details  of  each  formulation  used  will  be  presented  in  the 
appropriate  chapters  of  this  dissertation.  The  Investment  metrics  used  in  this  work  simply 
provides  a  means  for  quantifying  uncertainty  reduction  costs  in  the  absence  of  actual  cost 
data  and  can  be  easily  extended  to  include  a  function  or  set  of  functions  that  accurately 
quantify  uncertainty  reduction  costs  if  available.  Since  the  Investment  metric  quantifies 
the  cost  associated  with  uncertainty  reduction,  it  is  typically  a  function  of  the  uncertainty 
reduction  vector  a.  In  some  chapters  of  this  dissertation  the  Investment  metric  is 
abbreviated  by  the  single  character  I  in  the  interest  of  simplicity. 
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2.9  MULTI-OBJECTIVE  SENSITIVITY  ANALYSIS  (MOSA) 


Much  of  the  work  presented  in  this  dissertation  is  an  extension  of,  or  inspired  by  an 
approach  developed  by  Li  et  ah,  [2009a]  called  multi-objective  sensitivity  analysis 
(MOSA),  which  is  an  uncertainty  reduction  technique.  MOSA  uses  a  bi-objective 
optimization  problem  to  determine  an  optimal  reduction  in  parameter  uncertainty  at  a 
minimal  cost,  treating  the  uncertainty  reduction  vector  as  decision  variables. 

min  RAa ) 

a 

(2.3) 

min  Investment  {a) 

a 

Eqn.  (2.3)  can  be  solved  using  any  multi-objective  optimization  technique; 
however,  Li  et  al.  [2009a]  used  genetic  algorithms  with  much  success.  This  formulation 
elegantly  places  objective  function  variation,  Rf,  in  competition  with  the  cost  of 
producing  variation  reduction  Investment  (or  I  if  the  simplified  notation  for  Investment  is 
used),  and  produces  a  Pareto  set  of  solutions  for  the  designer  to  choose  from  based  on 
available  funds  for  uncertainty  reduction  and  objective  function  performance  needs. 
However,  the  MOSA  approach  is  limited  to  single  disciplinary  systems.  Furthermore, 
MOSA  provides  no  assurance  of  feasibility  under  uncertainty,  nor  does  it  provide  a 
means  for  achieving  feasibility  if  necessary. 

2.10  SURROGATE  APPROXIMATION  AND  KRIGING 

Surrogate  approximation,  also  widely  known  as  meta- modeling,  describes  a  set  of 
techniques  for  using  limited  actual  analysis  data  (often  selected  via  a  design  of 
experiments  (DOE)  technique)  to  approximate  a  complex  computational  model  [e.g.  Shan 
and  Wang,  2008;  Allaire  and  Willcox,  2008;  Wang  and  Shan,  2007].  Once  properly 
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verified  using  the  underlying  analysis  model,  a  meta-model  can  be  used  in  lieu  of  the 
actual  analysis  model  in  an  optimization  or  sensitivity  analysis  algorithm  to  conserve 
computational  effort.  There  are  numerous  techniques  that  can  be  used  to  approximate 
actual  analysis  data  and  one  such  particularly  useful  technique  used  in  this  work  is  a 
method  borrowed  from  geostatistics  called  kriging  [Martin  and  Simpson,  2005],  Kriging 
is  well  studied  and  none  of  the  work  presented  in  this  dissertation  attempts  to  improve 
kriging  specifically,  or  meta-modeling  in  general,  in  any  way.  Rather  this  work  simply 
uses  the  kriging  technique  as  a  means  for  increasing  the  computational  efficiency  of  the 
various  approaches  presented  in  this  dissertation.  Kriging  was  selected  over  other 
approximation  techniques  for  three  distinct  reasons:  1)  the  kriging  algorithm  ensures  than 
any  approximate  function  will  honor  all  observed  data  points,  2)  the  kriging  algorithm  is 
well  suited  to  high  dimensional  data  sets,  and  3)  the  kriging  algorithm  produces  an 
estimate  of  the  error  for  any  interpolated  point,  information  that  can  be  used  to  correct  the 
approximation  regularly  to  ensure  accuracy.  It  should  be  noted  that  meta-model  accuracy 
is  especially  important  in  a  sensitivity  analysis  context  as  inaccurate  approximations 
could  lead  to  errant  sensitivity  information.  This  issue  will  be  specifically  addressed 
when  necessary  throughout  this  dissertation. 
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CHAPTER  3:  MULTI-DISCIPLINARY,  MULTI-OUTPUT 

SENSITIVITY  ANALYSIS  (MIMOSA) 

3.1  INTRODUCTION  TO  RESEARCH  THRUST  1 

The  first  research  thrust  of  this  dissertation  involved  the  development  of  a  new 
global  sensitivity  analysis  approach.  This  new  approach  can  be  used  for  the  sensitivity 
analysis  of  multiple  sub-system  multi-output  analysis  models  in  which  the  inputs  to  the 
analysis  sub-systems  have  reducible  uncertainty.  This  new  approach  is  called  Multi¬ 
disciplinary  Multi-Output  Sensitivity  Analysis  (MIMOSA).  MIMOSA  has  several  key 
properties  and  capabilities:  a)  it  quantifies  variation  in  multiple  outputs  in  the  system 
and/or  each  sub-system  with  respect  to  input  interval  uncertainty,  considering  multiple 
uncertain  input  parameters  for  a  single  design  or  a  set  of  designs;  b)  it  identifies  the 
uncertain  parameters,  at  the  system  and  sub-system  levels,  which  have  the  greatest  effect 
on  system  output  variations;  and  c)  it  does  not  require  gradient  information  or  probability 
density  functions  to  quantify  the  uncertain  parameters,  rather  it  uses  interval  uncertainty 
which  is  more  flexible  and  easier  to  obtain  or  estimate. 

In  the  context  of  Multi-Disciplinary  Optimization  (MDO),  there  have  been 
numerous  approaches  proposed  in  the  literature  for  ensuring  that  optimal  design  solutions 
are  insensitive  to  the  uncertainty  associated  with  the  system’s  input  parameters,  mostly  in 
the  area  of  robust  optimization  (e.g.,  [Du  and  Chen,  2002  and  2005;  Smith  and 
Mahadevan,  2005;  Kokkolaras  et  al.,  2006;  Padula  et  al.,  2006;  Gu  et  al.,  2006; 
Chiralaksanakul  and  Mahadevan,  2007;  Li  and  Azarm,  2008]).  Those  approaches,  while 
valuable,  do  not  address  the  possibility  that  in  some  cases  the  uncertainty  in  a  system  or 
its  corresponding  sub-systems  could  be  reduced  to  some  level,  given  enough  investment, 
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and  that  reduction  in  uncertainty  could  in  turn  improve  both  the  system’s  and  its 
corresponding  sub-systems’  performance  without  the  need  for  additional  design 
optimization.  This  fact  drives  the  need  for  a  better  understanding  of  the  nature  and  effects 
of  uncertainty  in  multi-disciplinary  systems.  Such  knowledge  would  also  lead  to  a  better 
understanding  of  uncertainty,  the  relative  importance  of  particular  parameters  and  the 
opportunities  to  systematically  apply  limited  resources  in  order  to  reduce  the  variations  in 
a  system’s  outputs  optimally.  Sensitivity  Analysis  (SA)  techniques  provide  a  particularly 
useful  option  for  understanding  the  effect  of  uncertainty  and  the  application  of  SA 
techniques  to  uncertainty  reduction  problems  is  fast  becoming  an  area  of  interest  in 
current  research  efforts. 

The  typical  way  that  SA  approaches  are  currently  being  used  by  designers  is  as 
methods  for  understanding  the  effects  of  system  input  uncertainty  on  system  outputs 
[Iman  and  Helton,  1988;  Saltelli  et  al.,  2000].  The  majority  of  the  previous  works  in  this 
particular  application  of  SA  have  focused  on  systems  where  the  uncertainty  of  input 
parameters  has  a  presumed  probabilistic  distribution  (e.g.,  [Saltelli  et  al.,  2000;  Chen  et 
al.,  2005]),  or  on  systems  where  only  one  output  at  a  time  is  considered  (e.g.,  [Sobol, 
2001;  Helton  and  Davis,  2003]).  Moreover,  most  of  the  current  SA  approaches  are  only 
applicable  to  single-disciplinary  analysis-based  engineering  problems  [Saltelli  et  al., 
2000]  as  mentioned  in  Chapter  1.  In  contrast  to  these  single-disciplinary  methods,  SA 
approaches  that  treat  multi-disciplinary  systems  in  a  more  flexible  multiple  sub-system 
fashion  are  obviously  more  attractive  as  they  can  determine  the  sensitivity  of  sub¬ 
system’s  outputs  more  flexibly  and  capture  the  interaction  effects  between  sub-systems 
more  accurately.  Approaches  addressing  SA  in  a  multi-disciplinary  framework  remain 


30 


rare  in  the  literature  and  all  of  them  require  probabilistic  distributions  for  uncertainty  [Gu 
et  al.,  1998;  Yin  and  Chen,  2008],  or  use  gradient  information  of  the  function  under 
consideration  [Sobieszczanski-Sobieski,  1990;  Sobieszczanski-Sobieski  et  al.,  1991; 
Wehrhahn,  1991;  Noor  et  al.,  2000].  Moreover,  in  all  the  approaches  currently  in  the 
literature,  only  one  output  is  considered  for  each  sub-system.  However,  almost  all 
analysis  models  for  engineering  systems  are  multi-output  in  nature  and  probability 
distributions  for  uncertain  inputs  are  not  always  known  or  valid  (e.g.,  [Rao  and  Cao, 
2002;  Wu  and  Rao,  2007;  Li  and  Azarm,  2008;  Qiu  et  al.,  2008]). 

A  single-disciplinary  uncertainty  reduction  method,  called  Multi-Objective 
Sensitivity  Analysis  (MOSA),  [Li  et  al.,  2009a],  was  recently  developed  and  the  details 
of  this  approach  are  presented  in  Chapter  2.  Much  like  GSA  methods,  the  MOSA 
approach  can  determine  the  sensitivity  of  output  variations  to  multiple  input  parameter 
uncertainty.  However,  as  pointed  out  in  Chapter  2,  MOSA  did  not  address  the  uncertainty 
reduction  problem  at  each  sub-system  in  a  multi-disciplinary  framework.  Uncertainty  in 
the  input  parameters  of  a  multi-disciplinary  engineering  analysis  system  is  not  only 
unavoidable  but  also  results  in  variations  in  the  outputs  at  both  the  system  and  sub-system 
levels.  Moreover,  in  a  multi-disciplinary  analysis  model,  uncertainty  reduction  in  one 
parameter  (or  combination  of  parameters)  may  greatly  improve  the  performance  of  one 
sub-system,  but  may  have  no  effect,  or  perhaps  even  adverse  effects  on  other  coupled 
sub-systems.  For  this  reason,  it  is  necessary  that  SA  techniques  be  extended  to  multi¬ 
disciplinary  problems  with  multiple  outputs  in  order  to  consider  the  complex 
relationships  between  input  uncertainty  and  outputs  of  the  analysis  in  each  sub-system 
and  across  coupled  sub-systems. 
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MIMOSA  is  capable  of  determining  which  sub-systems  are  most  sensitive  to  input 
uncertainty  and  then  can  determine  how  limited  resources  should  be  applied  to 
uncertainty  reduction  in  order  to  obtain  a  maximum  reduction  in  output  variations  at  both 
the  system  and  sub-system  levels.  The  parameter  uncertainty  that  MIMOSA  is  concerned 
with  is  assumed  to  be  reducible  and  exists  not  only  as  input  in  each  sub-system  but  also 
in  the  couplings  between  sub-systems.  However,  MIMOSA  does  not  require  probability 
distribution  information  for  uncertainty  characterization,  but  instead  assumes  that  an 
uncertain  interval  is  known  for  each  uncertain  parameter.  In  the  MIMOSA  approach, 
given  the  known  interval  uncertainty  for  input  parameters  in  both  the  system  and  sub¬ 
system  levels,  a  designer  can  identify  parameters  whose  uncertainty  should  be  reduced  or, 
perhaps  even  eliminated,  in  order  to  achieve  the  desirable  reduction  in  variation  in  the 
system’s  and  it’s  corresponding  sub-system’s  outputs  simultaneously.  In  addition,  the 
relative  importance  of  each  of  the  sub-systems  can  also  be  determined  using  the 
information  produced  by  the  MIMOSA  approach.  Two  scalar  metrics  are  used  to  quantify 
the  amount  of  variation  in  outputs  and  the  cost  to  the  designer  of  reducing  the  uncertainty 
in  the  input  parameters  in  both  system  and  sub-system  levels.  A  bi-objective  SA  problem 
is  formulated  in  the  system  level  problem  and  in  each  sub-system  to  minimize  the 
variation  in  the  corresponding  system’s  outputs  while  simultaneously  minimizing  the 
investment  required  to  produce  the  necessary  reduction  in  the  uncertainty  of  the  system’s 
parameters.  A  numerical  example  and  an  engineering  design  example,  each  having  two  or 
three  sub-systems,  respectively,  are  used  to  demonstrate  the  applicability  of  the 
MIMOSA  approach. 
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The  rest  of  this  chapter  is  organized  as  follows.  Some  terminology  and  definitions 
used  are  described  in  Section  3.2.  Details  of  the  MIMOSA  approach  are  presented  in 
Section  3.3.  Two  examples,  a  numerical  example  having  two  sub-systems  and  an 
engineering  example  having  three  sub-systems,  are  presented  in  Section  3.4  to  illustrate 
the  applicability  of  the  approach.  Concluding  remarks  are  presented  in  Section  3.5.  A 
portion  of  this  chapter  was  presented  in  Li  et  al.  [2009b], 

3.2  BACKGROUND  AND  TERMINOLOGY 


3.2.1  Tolerance  Region 

The  concept  of  a  parameter  uncertainty  was  first  presented  in  Chapter  2  of  this 
dissertation.  The  region  of  the  parameter  space  bound  by  the  uncertain  input  parameter 
intervals  for  a  candidate  design  of  a  system  can  be  called  a  Tolerance  Region  (TR). 
Figure  3.1  below  depicts  a  TR  for  a  notional  two  parameter  system.  Recall  that  the  TR 
for  an  analysis  model  of  a  design  quantifies  the  amount  of  uncertainty  associated  with  the 
model’s  input  parameters.  These  are  not  selectable  values,  but  are  instead  a  property  of  a 
design  under  consideration  that  is  known  to  possess  uncertain  input  parameters.  As 
stated  in  Chapter  2,  the  parameter  uncertainties  considered  in  this  work  are  assumed  to  be 
defined  by  intervals  and  are  assumed  to  be  reducible. 


P  2 


Tolerance  Region 


Po.i±  A?,  px 


Figure  3.1:  Tolerance  Region 
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3.2.2  Uncertainty  Reduction  and  Retained  Tolerance  Region  (RTR) 

The  uncertainty  reduction  vector  a  =  { OC\,...,aK}  was  also  first  presented  in  Chapter 
2.  Recall  that  the  elements  of  the  a  vector  are  bound  by  0  and  1,  for  k  =  1 K,  one  for 
each  corresponding  uncertain  parameter.  The  term  Retained  Tolerance  Region  (RTR)  is 
used  to  describe  the  reduced  uncertain  region  of  the  parameter  space  defined  as  the 
portion  of  an  original  TR,  determined  by  the  a  vector  and  the  original  parameter 
uncertainty  Essentially,  RTR  can  be  any  symmetric  hyper-rectangle  that  is  inside  the 
original  tolerance  region  centered  at  the  nominal  parameter  values.  Recall  that  the  a 
vector  is  a  measure  of  the  prescribed  level  of  uncertainty  reduction  for  the  given 
uncertain  parameters  and  is  treated  as  a  set  of  the  decision  variables  and  will  be  selected 
within  the  MIMOSA  algorithm,  not  by  the  designer  beforehand. 

3.2.3  Reduced  Output  Sensitivity  Region  (ROSR) 

Regardless  of  which  type  of  uncertainty  (irreducible  or  reducible)  is  considered,  it 
is  possible  to  map  the  effects  of  input  parameter  variation  for  one  or  more  designs  under 
consideration  into  the  output  space  and  then  evaluate  those  effects  on  the  resulting 
uncertainty,  in  a  multi-output  sense  as  described  in  Chapter  2.  Given  a  known  TR,  which 
is  characteristic  of  a  given  design  and  its  associated  uncertain  parameters  (the  left  side  of 
Figure  3.2),  the  result  of  mapping  that  input  uncertainty  to  the  system  outputs  forms  an 
Output  Sensitivity  Region  (OSR)  (the  right  side  of  Figure  3.2)  for  each  design  under 
consideration.  Figure  3.2  assumes  a  two-input,  two-output  analysis  model  for  several  trial 
designs  in  the  interest  of  visualization,  but  the  concept  is  applicable  to  a  problem  of  any 
size. 
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Figure  3.2:  Mapping  from  the  Tolerance  Region  to  the  OSR 

For  a  trial  design  xo,  the  nominal  values  of  the  M  analysis  outputs  are  o(xo,  po)  = 
{e>i(xo,  Po),---,  om(xoJJo)}-  Output  variations  of  xo  will  be  considered  to  be  caused  by  the 
retained  parameter  variations  a  -  Ap  (as  originally  defined  in  Chapter  3)  in  Eqn.  (3.1): 

A o(x0,  p)  =  o(xa,  p)  -  o(xt,  p0) 

where  (3.1) 

p„  -  a  o  Ap  <  p  <  p0  +  a  o  Ap 


Notice  that  the  TR  of  parameters  on  the  left  of  Figure  3.2  can  lead  to  different 
OSRs  for  the  designs  represented  in  the  output  space  on  the  right  of  Figure  3.2.  This 
mapping  of  uncertain  parameters  can  be  quantified  by  measuring  the  distance  of  the 
largest  deviation  from  the  nominal  output  function  values  under  the  uncertain  parameter 
intervals  using  a  I  ML  norm.  This  quantification  was  first  presented  in  Chapter  2  and  is 
here  denoted  by  R ,  or  the  Reduced  OSR  (ROSR)  for  xo,  the  design  under  consideration 
for  analysis,  as  shown  in  Eqn.  (3.2).  This  R  metric  is  a  general  metric  (it  does  not 
differentiate  between  objective  and  constraint  functions)  since  in  the  context  of  this  work 
the  output  functions  under  consideration  may  be  either  objectives,  constraints  or  both. 
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R(  a)  =  max 

P 


A  o{p) 


Ao„ 


where 

Ao(p)  =  o(x0,p)-o(x0,p0 ) 
pn  -  a  °  Ap  <  p  <  p0  +  a  °  Ap 


(3.2) 


In  Eqn.  (3.2),  Aoo  is  the  Acceptable  Output  Variation  Region  (AOVR)  for  the  M 
outputs.  The  AOVR  value  is  specified  by  the  designer  and  then  the  variation  in  those 
outputs  is  normalized  by  the  corresponding  AOVR  values.  The  value  of  the  AOVR  is 
usually  assumed  by  the  designer  who  has  good  knowledge  about  the  system  or  a  clear 
requirement  on  the  acceptable  variation  range  of  the  system  performance.  Given  an 
AOVR,  it  is  required  that  the  variation  in  the  outputs  to  be  less  than  the  AOVR.  As  a 
result,  R  should  be  less  than  or  equal  to  1  for  the  acceptable  variation  for  the  outputs 
obtained  from  the  analysis  of  a  design  jco.  If  an  AOVR  is  not  available  or  not  desired,  R 
can  still  be  normalized  instead  by  using  the  nominal  output  values.  Normalization  in  Eqn. 
(3.2)  is  usually  necessary  depending  on  the  applications,  especially  for  the  engineering 
analysis  problems  that  have  multiple  outputs  with  different  units  or  orders  of  magnitude. 

As  shown  in  Figure  3.3,  a  measure  of  output  variation  R  can  be  reduced  (for 
multiple  designs,  as  shown)  by  reducing  the  RTR  for  a  given  set  of  input  parameters  as,  R 
is  a  function  of  the  corresponding  a  vector  values  for  the  system  (or  sub-system)  of 
interest.  This  is  concept  was  presented  conceptually  in  Chapter  2. 
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Figure  3.3:  Mapping  from  RTR  to  Reduced  OSR  (ROSR)  with  variation  R 


3.2.4  Correlation  Coefficient  Matrix  (CC) 


CC  is  a  matrix  of  correlation  coefficients,  calculated  for  a  matrix  whose  rows  are 
observed  data  of  variables  under  consideration  and  whose  columns  are  the  variables.  The 
(/,  y)th  element  of  the  matrix  CC  is  defined  as  shown  in  Eqn.  (3.3),  where  C(i,  j)  is  the 
covariance  value  between  variable  /  and  variable  j. 

C(i,j) 


CC(iJ) 


<JC(U)CU,j) 


(3.3) 


3.2.5  Investment 

The  Investment  metric,  presented  conceptually  in  Chapter  2,  is  essentially  a 
notional  function  that  correlates  the  amount  of  uncertainty  to  the  “cost”  required  to 
produce  that  uncertainty  reduction,  and  can  be  used  in  the  absence  of  actual  cost  data  or 
functions  for  an  engineering  analysis  model.  For  this  research  thrust,  the  Investment 
metric  used  is  the  formulation  first  presented  by  Li  et  al.  [2009a].  It  is  simply  defined  as 
the  normalized  representation  of  how  much  in  terms  of  both  perimeter  and  area  (or 
volume)  the  uncertain  region  in  the  parameter  space  for  a  set  of  parameters  is  reduced  as 
a  function  of  the  a  vector  values,  as  detailed  in  Eqn.  (3.4): 
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(3.4) 


Investment 


=  8, 


K 


V  J 


In  Eqn.  (3.4)  the  quantities  La*  and  IIa*represent  the  hyper-perimeter  and  hyper¬ 
volume,  respectively,  of  the  uncertainty  retained  in  the  parameter  space  after  the 
uncertainty  is  normalized  by  A p.  The  6  values  can  be  selected  and  aligned  according  to 
the  designer’s  preferences  on  the  hyper-perimeter  or  the  hyper-volume.  In  the  work 
presented  in  this  chapter  it  is  assumed  that  6\  =  #2  =  0.5,  meaning  that  both  the  volume 
and  perimeter  metrics  have  equal  weights  for  reduction.  The  hyper-perimeter  included  in 
Eqn.  (3.4)  indicates  that  the  investment  used  to  reduce  uncertainty  is  linear  to  the  amount 
of  uncertainty  in  each  parameter  and  identical  for  all  parameters  unless  a  weighting 
scheme  is  employed.  Additionally,  the  hyper- volume  included  in  Eqn.  (3.4)  describes  that 
the  investment  is  proportional  to  the  product  of  uncertainty  reduction  for  all  parameters, 
which  accounts  for  the  total  volume  of  uncertainty  reduction  given  a  set  of  a  values.  It 
can  be  seen  from  Eqn.  (3.4)  that  as  the  uncertainty  reduction  vector  elements  go  to  0, 
Investment  goes  to  1  indicating  that  a  maximum  possible  effort  is  required  to  eliminate  all 
the  uncertainty  in  the  input  parameters,  while  as  all  a*  values  go  to  1,  Investment  tends  to 
0  meaning  no  resources  are  required  to  reduce  the  input  uncertainty.  Clearly  Investment 
and  R  are  competing  metrics  as  smaller  output  variation  levels  will  always  require  greater 
cost.  It  should  be  noted  that  if  any  real  utility  or  cost  function  associated  with  parameter 
uncertainty  reductions  is  available  for  specific  applications,  such  as  a  known  dollar  value 
required  per  unit  of  machining  tolerance  improvement  for  a  geometric  dimension,  it  can 
be  easily  incorporated  into  Eqn.  (3.4). 
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3.2.6  Multi-Disciplinary  Multi-Output  Analysis  System 

Eqn.  (3.5)  and  Figure  3.4  depict  a  typical  multi-disciplinary  multi-output  system 
decomposed  into  three  coupled  sub-systems  that  share  some  design  variables  and  input 
parameters,  respectively  represented  as  x sh  and  psi„  along  with  their  local  sub-system 
variables  and  parameters  x ,•  and  /?„  i  =  1,  2,  3.  The  sub-system’s  outputs  can  be  single  or 
multiple  and  are  used  to  resolve  the  system  level  outputs.  In  Eqn.  (3.5),  yy  represents  a 
coupling  variable  vector:  Outputs  from  sub-system  i  (SSi)  and  inputs  to  sub-system  j 
(SS j).  The  vector  x,  and  o,  are  SS/’s  design  variables  and  outputs,  respectively.  The  vector 
Yj  in  Eqn.  (3.5)  represents  the  functions  that  are  used  to  calculate  the  coupling  variables 
yy.  Parameters  pt  represent  the  local  uncertain  parameters  that  have  interval  uncertainty. 
The  entire  system  outputs  can  be  represented  by  the  vector  o o,  which  can  be  assumed  to 
be  functions  of  shared  design  variables,  parameters,  and  local  (sub-system)  outputs, 
respectively, 

Given  the  shared  and  local  design  variables  xsh  and  Xj,  which  specify  the  candidate 
design(s),  the  variations  in  the  system  and  sub-system  outputs  are  dependent  on  the 
uncertain  input  parameters  and  coupling  variables. 

oi=0(xsh,psh,xi,pi,yJi) 

where  (3.5) 

y9  ^(xA’P^x^Pry*)  i  =  l,2, 3 
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Figure  3.4:  A  Multi-Disciplinary  System  with  Three  Sub-Systems 


One  typical  method  to  hierarchically  decouple  a  fully  (two-way)  coupled  multi¬ 
output  multi-disciplinary  system  is  to  introduce  a  new  set  of  variables  at  the  system  level, 
indicated  in  Figure  3.5  as  t,  which  represent  target  values,  one  for  each  of  the  coupling 
variables  that  connect  the  sub-systems  [e.g.,  Aute  and  Azarm,  2006],  Each  sub-system 
(SS /)  uses  the  target  variables  to  perform  local  calculations  and  obtain  all  local  output 
values  (i.e.,  o„  and  y,y).  The  collaborative  consistency  of  the  system  is  maintained  by 
enforcing  a  consistency  constraint  in  each  sub-system.  This  consistency  constraint 
requires  that  the  value  of  each  element  of  y,y  converges  to  its  corresponding  target  value 
in  ty  in  the  deterministic  case.  These  new  variables,  t,  are  considered  as  design  variables 
to  be  given  along  with  the  shared  design  variables  xsh,  at  the  system  level  (SSO). 


System  Level 


SSI:  given*, 
Oi=0(xsh,xi,psk,px,t) 


Figure  3.5:  Multi-Output  Multi-Disciplinary  System  in  a  Decomposed  Framework 
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3.3  MIMOSA  APPROACH 


In  this  section  the  specifics  of  the  MIMOSA  approach  are  presented.  The  issue  as 
to  how  to  ensure  the  collaborative  consistency  of  a  multi-disciplinary  system  under 
uncertainty  is  addressed  in  Subsection  3.3.1.  The  formulation  of  the  MIMOSA  approach 
is  outlined  in  Subsection  3.3.2.  The  detailed  steps  of  MIMOSA  are  presented  in 
Subsection  3.3.3. 

3.3.1  Collaborative  Consistency  of  MIMOSA 

As  mentioned  previously,  in  order  to  decouple  the  multiple  sub-systems  that  make 
up  a  multi-disciplinary  system  and  maintain  the  collaborative  consistency  of  the  system,  a 
target  variable  for  each  coupling  variable  must  be  introduced  at  the  system  level  and  then 
used  in  each  sub-system.  However,  when  uncertainty  is  considered  in  the  system  the 
collaborative  consistency  of  the  system  is  not  straightforward  and  deserves  more 
attention  [Li  and  Azarm,  2008].  A  multi-disciplinary  system  is  said  to  possess 
collaborative  consistency  when  all  sub-systems  achieve  the  same  value  for  each  coupling 
variable  within  the  system’s  decomposed  framework,  thus  ensuring  that  all  sub-systems 
will  work  together  consistently  to  define  a  design. 

When  considering  the  sub-system  couplings  shown  in  Figure  3.4,  the  propagation 
of  uncertainty  implies  that  outputs  from  one  sub-system  are  not  only  affected  by  the 
uncertainty  from  this  sub-system’s  parameters  but  also  by  the  uncertainty  in  coupling 
variables.  With  the  introduction  of  interval  uncertainty  in  parameters,  the  collaborative 
consistency  constraint  can  no  longer  be  satisfied  since  the  uncertainty  in  the  input 
parameters  p\  of  SS  i  leads  to  a  range  rather  than  a  single  value  for  each  component  in  the 
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coupling  variables  yy.  The  collaborative  consistency  constraint  in  SSz  cannot  force  a 
range  of  output  values  for  each  coupling  variable  yy  to  converge  to  a  single  target  value 
of  ty.  As  a  result,  a  system  and  corresponding  sub-systems  could  become  inconsistent  due 
to  a  mismatch  between  the  output  range  of  each  coupling  variable  and  the  deterministic 
value  of  each  target  variable.  To  resolve  this  mismatch  the  target  variables  t  must  be  able 
to  tolerate  the  resulting  variation  ranges  in  the  coupling  variables  y.  In  other  words,  to 
accept  the  variation  in  the  coupling  variables  y,  target  variables  t  should  have  an 
associated  tolerance  range.  Given  a  candidate  design,  as  long  as  the  established  tolerance 
range  of  ty  encloses  the  resultant  variation  in  y-y  for  the  system,  the  variations  in  coupling 
variables  y-y  can  be  absorbed  by  a  local  uncertainty  associated  with  the  targets  ty  for  SS/, 
ensuring  that  the  system  will  remain  collaboratively  consistent. 

To  accommodate  the  propagation  of  uncertainty,  it  is  necessary  to  ensure  that  the 
selected  tolerance  regions  associated  with  the  target  variables  are  large  enough  to  enclose 
the  expected  variations  in  the  corresponding  coupling  variables  in  order  to  maintain  the 
feasibility  of  the  entire  system.  This  tension  between  uncertainty  reduction  and 
collaborative  consistency  is  resolved  in  the  MIMOSA  approach  by  adding  a  new 
collaborative  consistency  constraint  under  uncertainty  in  each  sub-system.  In  order  for 
this  new  constraint  to  function  properly  and  have  the  desired  result  of  ensuring  the  overall 
system  consistency,  all  target  variables  which  are  a  part  of  the  collaborative  consistency 
constraints  are  considered  to  have  interval  uncertainty  at  the  system  level  and  have 
corresponding  a  vector  values  selected  to  control  the  associated  uncertainty.  Using  this 
method,  as  long  as  the  resultant  variation  in  the  coupling  variables  output  from  each  sub¬ 
system  is  less  than  the  retained  uncertainty  in  the  target  variables  passed  down  from  the 
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system  level,  the  collaborative  consistency  among  sub-systems  will  be  ensured  (see  the 
next  subsection  for  a  detailed  formulation).  The  selection  on  the  original  TR  of  the  target 
variabls  are  dependend  on  the  designer’s  opinion  or  specific  requirments  on  couplings. 

3.3.2  Formulation  of  MIMOSA 

For  simplicity,  the  formulation  of  the  MIMOSA  approach  will  be  presented  as 
applied  to  a  single  candidate  design,  but  extending  this  approach  to  multiple  candidate 
designs  is  straightforward  to  accomplish  (see  (Li  et  al.,  2009a]).  The  MIMOSA  approach, 
as  shown  in  Figure  3.6  consists  of  performing  a  two-objective  optimization  problem  for 
SA  at  the  system  level  and  for  each  sub-system  in  a  decomposed  framework.  The  SA 
optimization  problem  is  used  to  determine  the  best  R  and  Investment  attainable  as  a 
function  of  the  a  vector  elements  for  uncertain  parameters  p  and  the  target  variables  t. 
This  is  conducted  at  the  system  level  and  in  each  of  the  sub-systems  being  considered  in  a 
bi-level  fashion.  The  formulation  assumes  that  the  designer  already  has  obtained  a 
candidate  design  (or  designs)  in  which  input  parameters  have  reducible  interval 
uncertainty. 

At  the  system  level  (SSO),  the  variables  are  the  vector  aSso,  which  includes  a.,/,  and 
at  for  psh  and  t,  respectively.  The  two  objectives  of  the  SA  optimization  at  the  system 
level  are  1)  to  minimize  Investment  and  2)  to  minimize  the  variation  in  the  system  level 
design  outputs  represented  as  Rs so,  as  shown  in  Eqn.  (3.6).  Here,  Rsso  at  the  system  level 
is  defined  as  the  ll.lloo  norm  of  the  variation  in  the  corresponding  system  outputs. 
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min  Rsso  =  |AoSS0|L 
min  Investment^ 

ass, 

where 

AoSS0  =  o„(x0, />)-<>„  (*„,/>„) 

P 0  -  «sso  °  Ap  <  p  <  p„  +  «sso  o  Ap 
«sso  ={«*»>«,} 

In  Eqn.  (3.6),  Ao$so  is  the  variation  in  SSO’s  outputs.  If  the  outputs  in  SSO  have 
different  units  or  different  orders  of  magnitude  with  respect  to  each  other,  the 
normalization  of  the  variation  in  the  system  level’s  (SSO)  outputs  is  necessary.  The  most 
obvious  choice  is  to  normalize  Aosso  by  the  nominal  output  values  for  a  candidate  design 
under  consideration. 

The  formulation  of  the  sub-system  SA  problem  is  very  similar  to  the  system  level, 
with  the  key  addition  of  the  need  to  ensure  collaborative  consistency  of  the  design  under 
uncertainty.  To  do  this  consistency  check  under  uncertainty,  in  each  sub-system  level 
optimization  problem,  SSi,  the  quantity  7]c  is  defined  as  the  maximum  IHloo  distance  from 
the  coupling  variable  yy  to  the  nominal  target  variable  % o,  as  shown  in  Eqn.  (3.7a).  As 
long  as  this  distance  is  within  the  optimizer-specified  retained  tolerance  region  a  °  Atyp 
for  target  variables,  the  variation  in  the  coupling  variable  is  acceptable. 

r/c.  =  max  ||C  J|  ,  i  =  1,  2,  3  (3.7a) 

I  p  t  °° 

Eqn.  (3.7b)  below  details  the  value  C„  used  in  Eqn.  (3.7a),  which  quantifies  the 
difference  between  y,j  and  the  nominal  value  of  ty,  normalized  by  the  RTR  of  ty.  Recall 
that  a  » A ty  is  the  retained  tolerance  region  for  target  variables  ty,  in  which  a  is  determined 
by  the  system  level  optimizer  and  sent  to  SS/  from  SSO: 
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c  =  yAt’P’tj.)  -  ty.o 
a  o  At,{ 


,  j  *  i 


p  0  -  a  °  Ap  <  p  <  p  0  +  a  °  Ap 
t  a  o  A  t  <  t  <  t  ...  +  a  o  A  t 

ji  ,  U  ji  ji  ji  ,  U  ji 

P  =  {Psh  .  P,}>  «  =  {«SS0  *«|} 


(3.7b) 


For  the  SA  problem  in  sub-system  i  (SS /),  the  decision  variables  are  the  uncertainty 
reduction  vector  a,  for  the  local  uncertainty  parameters  p,.  The  two  objectives  for  each 
sub-system  are  again  to  minimize  Investment $s;  while  simultaneously  minimizing  the 
maximum  of  either  the  output  variation  at  the  local  sub-system  level  Rssi ,  or  the  variation 
in  coupling  variables,  r/a  in  SS i.  In  Eqn.  (3.8),  7?ss/is  again  a  measure  of  the  variation  in 
the  sub-system’s  outputs,  here  normalized  by  the  corresponding  sub-system’s  AOVR 
AOo j,  which  is  the  presumed  acceptable  variation  for  the  sub-system’s  outputs.  This 
optimization  problem  is  subject  to  two  constraints  as  shown  in  Eqn.  (3.8),  including:  1) 
the  variation  in  the  SS/’s  original  outputs  must  be  enclosed  by  the  designated  AOVR  of 
SSz;  and  2)  the  variation  in  coupling  variables  must  be  less  than  the  retained  tolerance 
region  of  target  variables  (determined  in  SSO).  These  constraints  must  be  satisfied  in  all 
sub-systems  in  order  to  maintain  the  consistency  of  the  system  under  uncertainty.  Thus 
the  SA  optimization  problem  formulation  in  SS/  is  as  follows: 
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min  max.(RSSi,ijc) 


min  Investments 


s.t. 


max  ( RSSt ,  tjc  )  <,  1 
A ot(p) 


/?„,  =  max 

p.‘. 


A o 


O.i 


nr  =  max||C,|| 
*<>  . - 


where 

C  = 


«  o  A/„ 


Aoi(p)  =  oi(x0,p)-oi(x0,pJ 

p0-a°Ap£  p<i  p0+a°Ap 

t  -a°At  <tt  <>t  +«oAf 
P={P1*^(K«  =  {«SS.’«(} 


(3.8) 


The  MIMOSA  formulation  is  also  shown  in  Figure  3.6  at  both  system  and  sub-system 
levels. 


System  Level  Sub-System  Level 


Figure  3.6:  MIMOSA  Formulation 

A  simplified  formulation  of  MIMOSA  at  the  sub-system  level  is  also  provided  as 
shown  in  Eqn.  (3.9),  for  the  cases  where  a  designer  does  not  know  or  specify  an  AOVR 
for  the  sub-system  outputs.  In  this  regard,  as  long  as  the  overall  system  is  collaboratively 
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consistent  by  enforcing  the  constraint  on  Tja  in  Eqn.  (3.9),  the  solution  found  will  be 
acceptable.  In  this  simplified  formulation,  normalization  of  i?ss/  is  still  necessary  in  order 
to  handle  the  possibility  of  different  units  or  orders  of  magnitude  in  the  sub-system 
outputs,  and  is  accomplished  by  using  the  nominal  output  values.  However,  the  value  of 
Rssi  is  usually  much  less  than  the  value  of  rja  in  this  formulation  and  as  a  result  the  first 
objective  of  the  SA  problem  for  each  sub-system  should  only  minimize  Rssi  alone.  In 
Section  3.4,  Eqn.  (3.8)  and  Eqn.  (3.9)  will  be  used  in  the  numerical  and  engineering 
example,  respectively. 


min  R „c. 

a,  SS‘ 

min  Investment 

a. 

s.t. 

nCi  ^ 1 

A o,  (p) 

ot(x0,p0) 

where 


RSSi  =  max 

p,t„ 


1c, 


=  max 

p,‘f 


C  _  .V 

acto, 

A o,.  (p)  =  o ,  (x0 ,  p)  -  ot  (x0 ,  p0 ) 

p0-a°Ap<p<p0+a°Ap 
t ...  -  a  o  At  <t  <  t  „  +  a  o  At 

J',0  J<  ]‘  JI.  o  Ji 

P  =  {PA,P, },<*  =  {<*  sso»a/} 


(3.9) 


Notice  first  that  in  Eqn.  (3.6)  for  SSO  and  Eqn.  (3.8)  (or  Eqn.  (3.9))  for  SS/,  no 
additional  information  on  the  input-output  relation  from  the  analysis  models  is  required. 
The  analysis  models  in  SSO  and  SS/  are  treated  like  black-boxes  and  can  be  any  kind  of 
functions  or  computer  simulations  as  long  as  the  input  and  output  values  are  provided.  In 
addition,  if  one  oq  value  in  the  uncertainty  reduction  vector  is  intentionally  fixed  to  be 
zero  and  all  other  elements  to  be  one,  one  parameter  is  essentially  “left  out”  and  the 
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corresponding  Rssi  value  can  be  calculated.  By  repeating  this  for  each  parameter  (or 
combination  of  parameters),  the  quantitative  comparison  on  the  importance  of  the 
parameter(s)  can  be  obtained  as  using  a  traditional  “leave  one  out”  GSA  method. 
However,  the  MIMOSA  approach  is  more  capable  in  that  it  allows  every  au  value  not 
only  to  be  zero  or  one  but  also  any  value  between  them,  optimally  determined  by  the 
approach.  In  this  regard,  all  uncertain  input  parameters  in  one  sub-system  can  vary  within 
the  RTR  specified  by  the  a  vector  and  can  affect  the  variation  in  sub-system  outputs 
simultaneously.  Thus,  not  only  the  main  effect  of  each  parameter,  but  also  the  interaction 
effects  of  those  parameters  are  considered  in  this  approach  within  the  optimal  uncertainty 
ranges.  The  individual  importance  of  each  parameter  can  also  be  determined  from  the 
resulting  optimal  solutions  to  Eqns.  (3.6)  and  (3.8)  as  discussed  later  in  Section  3.4.  Due 
to  the  system  properties  considered  by  the  MIMOSA  approach,  e.g.,  multi-input,  multi¬ 
output,  black-box  analysis  models,  and  most  importantly,  variable  uncertainty  ranges 
(i.e.,  not  just  zero  or  one  for  a  values),  the  MIMOSA  approach  is  capable  of  providing 
more  information  about  a  system  than  produced  by  the  other  existing  multi-disciplinary 
GSA  approaches  reviewed  in  Section  3.1. 

3.3.3  Steps  of  MIMOSA 

In  order  to  perform  the  MIMOSA  approach  described  above,  an  algorithm  was 
developed  and  a  step-by-step  description  of  that  algorithm  follows.  All  system  and  sub¬ 
system  SA  problem  optimizations  are  accomplished  using  evolutionary  algorithms  or 
more  specifically  using  the  Multi-Objective  Genetic  Algorithm  (MOGA)  [Deb,  2001]. 
The  usage  of  MOGA  to  solve  the  system  or  sub-system  level  problems  is  not  required 
and  other  multi-objective  optimization  approaches  could  also  be  applied,  if  applicable 
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and  desired.  However,  MOGA  is  used  in  this  approach  because  of  its  flexibility  in  finding 
all  Pareto  solutions  simultaneously,  its  ability  to  handle  non-linear  and  non-differentiable 
output  functions  with  both  continuous  and/or  discrete  variables,  and  its  ease  in 
incorporating  “black-box”  type  simulations.  These  properties  are  very  common  in  the  real 
engineering  applications  that  are  the  target  problems  of  this  approach.  However,  MOGA 
is  not  the  only  choice  of  optimization  solvers  for  use  in  this  approach.  The  decomposed 
SA  optimization  formulation  is  solved  using  a  multi-objective  multi-disciplinary 
optimization  technique  [Aute  and  Azarm,  2006]  as  previously  mentioned.  The  MIMOSA 
steps  are  as  follows: 

Step  1 :  Select  a  candidate  design  alternative  (or  a  trial  design),  jto,  whose  sensitivity 
analysis  is  to  be  studied. 

Step  2:  Select  which  input  parameters  are  to  be  studied,  and  then  determine  the 
interval  uncertainty  for  those  parameters,  specifying  the  original  TR,  p  =  [p0  +  A p,  po  - 
A p]  for  all  parameters  at  the  system  and  sub-system  levels. 

Step  3:  Select  the  original  TR  for  target  values  t  =[to+  A t,  to  -  At]  at  the  system  level. 

Step  4:  Initialize  the  optimization  problem  at  the  system  level  (SSO)  as  in  Eqn. 
(3.6),  given  the  system  level  design  xSh  and  original  TR  for psh,  and  t. 

Step  5:  Select  the  initial  values  for  aSso  at  the  system  level  for  each  element  in  psh 
and  t,  and  use  asso  to  calculate  Investments so- 

Step  5:  Send  the  system  design  variables,  nominal  shared  parameters,  nominal 
target  values,  and  system  level  aSso  values  to  each  sub-system  level  SA  problem. 
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Step  6:  Simultaneously  initialize  the  SA  approach  in  each  sub-system  level  problem 
(SS /),  given  the  xsh,  «sso,  Psh,  and  t  values  from  SSO  along  with  sub-system  variables  and 
parameters,  including  initial  a,  values  for  the  sub-system  level  uncertain  parameters. 

Step  7:  Determine  the  optimal  a,  to  the  optimization  problem  in  Eqn.  (3.8)  (or  Eqn. 
(3.9))  for  each  sub-system  using  MOGA,  considering  the  variation  in  sub-system’s 
outputs  (i.e.,  the  variation  in  outputs  and/or  coupling  variables)  and  Investment  as  the 
objectives,  while  ensuring  the  collaborative  consistency  constraint  and/or  AOVR 
constraint. 

Step  8:  Return  optimal  output  values  and  local  optimal  uncertainty  reduction  vector 
a,  to  the  system  level  (SSO)  from  the  sub-systems.  If  more  than  one  local  optimal  solution 
a,  is  identified,  some  selection  strategy  should  be  used  to  select  one  local  optimal  solution 
for  each  sub-system.  In  this  work,  two  selection  strategies  have  been  used:  the  optimal 
solution  from  the  SS/’s  Pareto  with  the  maximum  and  minimum  Rssi  value  will  be 
selected  as  the  sub-system’s  single  optimal  solution  in  the  numerical  example  and  the 
engineering  example  in  Section  3.4,  respectively,  and  will  be  returned  back  to  SSO.  Other 
appropriate  strategies  for  selecting  a  single  solution  from  the  sub-system  Pareto  are  also 
acceptable. 

Step  9:  Analyze  system  level  R  vs.  Investment  Pareto  optimal  solutions  to  Eqn. 
(3.6)  for  uncertainty  reduction  vector,  «Sso- 

Step  10:  Check  stopping  criteria  of  the  system  level  SA  problem.  If  stopping 
criteria  are  satisfied,  stop  the  algorithm;  otherwise  generate  a  new  set  of  candidates  of 
uncertainty  reduction  vector  asso  at  the  system  level  from  the  optimizer,  go  to  Step  5,  and 
repeat  sub-system  level  optimizations. 
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The  stopping  criteria  used  is  a  pre-specified  maximum  number  of  iterations  (which 
is  large  enough  to  ensure  convergence),  plus  an  additional  complementary  stopping 
criterion  that  further  requires  that  a  sufficient  number  of  Pareto  solutions  are  produced 
during  several  successive  generations.  In  other  words,  when  the  number  of  Pareto 
solutions  is  more  than  some  pre-specified  percentage  of  the  population  size  and  when  it 
becomes  steady  (e.g.,  the  number  of  Pareto  solutions  is  more  than  “40%  population  size”, 
for  several  generations),  it  can  be  concluded  that  the  algorithm  has  converged.  Compared 
to  single-disciplinary  SA  approaches,  the  computational  effort  in  this  new  MIMOSA 
approach  will  be  larger  due  to  the  bi-level  nature  of  the  problem  and  will  increase  further 
when  the  number  of  sub-systems  considered  is  increased,  since  a  SA  optimization 
problem  (as  shown  in  Eqn.  (3.8))  must  be  solved  for  each  sub-system  considered.  If  the 
number  of  function  calls  in  SSO  in  Eqn.  (3.6)  is  Nsso  and  the  number  of  function  calls  in 
each  sub-system  is  Nss;  and  the  number  of  function  calls  to  evaluate  Rssi  in  each  sub¬ 
system  is  Nr,  then  the  total  number  of  function  calls  in  MIMOSA  is 
0[NssoxNumSS(NssiXN/f)],  where  NumSS  is  the  number  of  sub-systems.  The 
computational  cost  of  MIMOSA  may  increase  when  the  number  of  sub-systems  and/or 
the  number  of  system  inputs  are  increased. 

3.4  EXAMPLES  AND  RESULTS 

In  this  section,  a  numerical  example  and  an  engineering  example  are  presented  to 
demonstrate  the  applicability  of  the  MIMOSA  approach.  Eqns.  (3.8)  and  (3.9),  as  well  as 
different  selection  strategies  discussed  above  in  Step  8  of  Subsection  3.3.3,  are  used  in 
those  two  examples,  respectively,  for  the  sub-system  level  SA  problems. 
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3.4.1  Numerical  Example 


A  two-output,  bi-level  numerical  example  is  adapted  from  a  previously  presented 
bi-level  MDO  problem  with  two  coupled  sub-systems  [Li  and  Azarm,  2008],  The  two- 
output  formulation  for  this  problem  in  a  single-disciplinary  (or  all-at-once)  formulation  is 
given  in  Eqn.  (3.10).  There  are  three  design  variables:  x  =  { jcj,  x2,  x3[,  two  output 
functions:  o  -  [o\.  o2). 


°\  =x2  +JC3 +y, +e  yi 

°2  =x\  +V*2  +(y22  — y/)/ 10*  +  150 


(3.10) 


The  two  coupling  variables  are  yi  and  y2  which  make  the  two  sub-systems  fully  coupled: 

y,  =y,(jr,y2)  =  x12  +  x2  -0.2y2  ,  . 


y2  =Y2  (x,  y,)  =  x,  +x3  + 


The  above  all-at-once  formulation  is  converted  into  a  system  level  and  two  sub¬ 
system  level  SA  problems.  Between  these  two  sub-systems  SSi,  i  =1  and  2,  there  are  two 
coupling  variables  y  =  { yi,  y2 } -  In  each  sub-problem,  there  are  two  outputs  in  addition  to 
the  coupling  variable.  The  decomposed  formulation  for  this  problem  is  as  follows. 

In  SS0,  xsh  includes  only  one  design  variable  jci  and  the  vector  of  target  variables  t 
includes  two  target  variables  t\  and  t2  corresponding  to  the  two  coupling  variables  yi  and 
y2.  Thus  xsh  =  {JCi } ,  t\2  =  [ri  }  and  t2\  =  { } -  Each  output  in  SS0  is  the  summation  of  the 
corresponding  outputs  of  two  sub-systems  as  shown  in  Eqn.  (3.12),  where  on,  i,  j  =1  and 
2,  is /h  output  in  SS/  and  calculated  from  Eqn.  (3. 1 3)  and  Eqn.  (3. 14)  below: 


(3.12) 
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SSI  has  the  local  design  variable:  x\  =  {*2},  and  the  local  copy  of  the  target 
variable  for  ti,  i.e.,  t-i  .  The  two  outputs  of  SSI  plus  the  coupling  variable  yi  are  given  in 
Eqn.  (3.13): 

Ox,  1  =*22  +yt 

°>.  2  =V^-y,3/104  +150  (3-13) 

y,  =  jc,2  +x2  -0.2t'2 

Similarly,  SS2  has  the  local  design  variable  xj  =  {*3}  and  the  local  copy  of  the 
target  variable  for  tu  i.e.,  t\2.  The  two  outputs  of  SS2  plus  the  coupling  variable  y>2  are 
given  in  Eqn.  (3.14): 

°2i  =x3+  e~n 

°2,2  =  xi  +  y22/l  04  (3-14) 

y2  = 

The  interval  uncertainty  is  assumed  in  input  variables  jci,  X2  and  *3,  within  +6% 
from  nominal.  The  AOVR  for  the  outputs  o\  and  02  in  SSI  and  SS2  is  ±5  units  from  their 
nominal.  The  original  TR  for  the  target  variables  is:  {Afi,  At2]  =  {±4,  +3}  units  from  their 
nominal.  The  candidate  design  alternative  selected  is  jc0:  {*1,  x2,  *3,  t\,  r2 }  =  {-5.478, 
0.035,  0.410,  30.262,  0.448}.  The  variables  at  the  system  level  are  asso  =  {&xv  0(tv  cCt2)- 
Here  it  is  assume  that  not  all  of  the  uncertainties  in  the  system  can  be  completely  reduced 
for  each  parameter,  so  oc,x  and  a^are  assumed  to  be  in  the  ranges  [0.3,  1]  and  [0,  1], 

respectively.  At  the  sub-system  level,  the  design  variable  in  SSI  and  SS2  are  a\  =  {c^2} 

and  a.2  =  {c^3}  which  are  in  the  range  [0,  1],  The  MIMOSA  formulation  of  this  example 

for  SS0,  SSI  and  SS2  is  shown  in  Figure  3.7.  The  Pareto  solutions  are  shown  in  Figure 
3.8. 
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sso 

™n  ^  =  144.1. 

min  Investment^, 

*». 

where 

=  o,,(x)-o„(x,) 
AOku  =0.Ax)-°ii(x,) 
I  =  {x,,x1,x1} 
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Figure  3.7:  Numerical  Example:  The  MIMOSA  Formulation 


Figure  3.8:  Numerical  Example:  Pareto  Solutions  at  (a)  SSO,  (b)  SSI  and  (c)  SS2 


In  this  example,  the  optimal  solution  from  the  SS/’s  Pareto  with  the  maximum  Rs$i 
value  is  selected  as  the  sub-system’s  single  optimal  solution  and  is  sent  back  to  SSO.  This 
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selection  strategy  is  used  in  the  cases  that  the  designers  in  the  sub-systems  prefer  the 


optimal  a,  with  the  minimum  investment  as  long  as  the  AOVR  and  consistency 
constraints  are  satisfied. 

The  obtained  Pareto  solutions  with  three  typical  a  solutions  at  the  system  and  sub¬ 
system  levels  using  the  formulation  in  Eqn.  (3.8)  are  shown  in  Figure  3.8  as  well  as  in 
Table  3.1.  The  trade-off  between  the  Investment  and  variation  in  SSO’s  outputs,  Rsso  are 
clear.  Rs so  (the  variation  in  SSO)  is  decreasing  with  the  increasing  of  Investment.  As 
shown  in  Table  3.1,  «m  =  {0.99  0.99  1}  gives  almost  the  original  uncertainty,  which 
represents  the  maximum  uncertainty  in  the  input  parameters.  Rsso(«iii)  is  approximately 
4%  of  the  nominal  output  values  of  SSO,  which  is  almost  the  biggest  variation  observed 
in  SSO.  When  the  RTR  is  reduced  to  «n  =  {0.91  0.39  1},  Rsso(«n)  is  reduced  to  1.5%  of 
the  nominal  output  values.  If  the  RTR  is  further  reduced  to  a\  (i.e.,  {0.02  0.78  0.96}), 
Rsso(ai)  is  only  about  0.9%  of  the  nominal  output  values,  which  is  the  smallest  variation 
observed  in  SSO  for  this  problem.  The  corresponding  sub-system  values  are  shown  in  the 
chapter  appendix.  Clearly,  as  Investment  values  are  increasing  in  this  procedure,  the 
amount  of  uncertainty  in  input  parameters  and  outputs  is  eliminated,  as  shown  in  Table 

3.1.  The  details  of  sub-system  solutions  corresponding  to  Figure  3.8  are  shown  in  Table 

3.2. 


Table  3.1:  Typical  a  Solutions,  Investment  vs.  Rsso 


X\ 

t\ 

ti 

Rsso  (%  of  nominal ) 

Investment 

a,\ 

0.02 

0.78 

0.96 

0.86% 

70% 

d\\ 

0.91 

0.39 

1.00 

1.49% 

44% 

am 

0.99 

0.99 

1.00 

3.94% 

2% 
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Table  3.2:  Detailed  Values  for  Pareto  Solutions  in  Figure  3.8 


(a)  SSI 


Solution  # 

a,2 

max{/?ss2,7c,} 

Investments si 

ncx 

A  0i,i 

Aoi  2 

1:  a\ 

0.996 

0.306 

0.004 

0.306 

0.648 

0.165 

2 

0.992 

0.237 

0.008 

0.237 

0.628 

0.16 

3 

0.985 

0.314 

0.015 

0.314 

0.839 

0.214 

4:  an 

0.99 

1 

0.01 

1 

1.273 

0.329 

5 

0.959 

1 

0.041 

1 

1.444 

0.373 

6 

0.996 

1 

0.004 

1 

1.969 

0.494 

7 

1 

1 

0 

1 

2.302 

0.569 

8 

0.996 

1 

0.004 

1 

2.625 

0.646 

9 

0.982 

1 

0.018 

1 

2.717 

0.662 

10 

0.859 

1 

0.141 

1 

2.849 

0.699 

11 

0.998 

1 

0.002 

1 

3 

0.728 

12 

0.998 

1 

0.002 

1 

3 

0.735 

13 

0.984 

1 

0.016 

1 

3.215 

0.779 

14 

0.998 

1 

0.002 

1 

3.5 

0.834 

15 

1 

1 

0 

1 

3.5 

0.838 

16 

1 

1 

0 

1 

3.5 

0.84 

17 

0.969 

1 

0.031 

1 

3.591 

0.851 

18:  am 

1 

1 

0 

1 

3.637 

0.865 

(b)  SS2 


Solution  # 

a,3 

max{/?ss2,7c2} 

Investment^! 

Vc2 

A<?2,1 

Ac>2,2 

l:ai 

0.948 

0.112 

0.052 

0.112 

0.217 

0.006 

2 

0.896 

0.145 

0.104 

0.145 

0.29 

0.006 

3 

0.968 

0.143 

0.032 

0.143 

0.281 

0.026 

4:  an 

0.889 

0.153 

0.111 

0.153 

0.216 

0.299 

5 

0.933 

0.166 

0.067 

0.166 

0.231 

0.324 

6 

0.979 

0.177 

0.021 

0.177 

0.239 

0.3 

7 

0.999 

0.199 

0.001 

0.199 

0.49 

0.324 

8 

0.889 

0.198 

0.111 

0.198 

0.266 

0.314 

9 

0.951 

0.209 

0.049 

0.209 

0.273 

0.327 

10 

0.951 

0.218 

0.049 

0.218 

0.277 

0.328 

11 

0.971 

0.214 

0.029 

0.214 

0.278 

0.32 

12 

0.85 

0.212 

0.15 

0.212 

0.283 

0.327 

13 

0.97 

0.23 

0.03 

0.23 

0.287 

0.328 

14 

0.952 

0.223 

0.048 

0.223 

0.291 

0.314 

15 

0.997 

0.219 

0.003 

0.219 

0.287 

0.302 

16 

0.896 

0.23 

0.104 

0.23 

0.296 

0.327 

17 

0.995 

0.233 

0.005 

0.233 

0.297 

0.327 

1 8:  am 

0.962 

0.231 

0.038 

0.231 

0.299 

0.327 
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In  order  to  identify  the  relative  importance  of  each  uncertain  parameter,  a 
correlation  plot  of  a.\,  ct2,  R  and  Investment  for  all  Pareto  a  solutions  in  Figure  3.8  is  given 
in  Figure  3.9.  To  clearly  illustrate  the  correlation  among  obtained  solutions  in  Figure  3.8, 
correlation  coefficient  matrix  values  were  calculated  as  defined  in  Section  3.2,  for  a,  /?, 
and  Investment  and  are  reported  in  Table  3.3.  It  can  be  shown  from  Figure  3.9  and  Table 
3.3  that  among  .Vi,  t\,  and  t2,  the  variable  .*1  has  the  strongest  correlation  to  the  variation  in 
SSO’s  outputs  and  Investment. 


a  _ 

.1 

•• 
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•  » 
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Figure  3.9:  Numerical  Example:  Plots  of  Correlations  among  a,  Investment  and  R 


Table  3.3:  Correlation  Coefficient  Matrix  of  Pareto  solutions  at  System  Level 


Correlation  Coefficient 

a,. 

a, 2 

R 

Investment 

a*\ 

1 

- 

- 

- 

- 

«/, 

-0.20 

1 

- 

- 

- 

«t2 

0.79 

-0.33 

1 

- 

- 

R 

0.78 

0.45 

0.54 

1 

-0.98 

Investment 

-0.87 

-0.29 

-0.64 

-0.98 

1 

Several  interesting  observations  can  be  concluded  from  Figure  3.8  and  Table  3.2:  i) 
The  variations  in  all  the  sub-system  level  outputs  and  couplings  are  less  than  the  specified 
AOVR  and  RTR  of  the  target  variables  in  each  sub-system,  but  the  variation  in  SSI  is 
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larger  than  the  variation  in  SS2  as  shown  in  Figures  3.8(b)  and  3.8(c),  showing  that  SSI 
is  more  sensitive  to  the  uncertainty;  ii)  Since  the  approach  selects  the  optimal  solution 
from  SS’s  Pareto  with  the  minimum  Investment ,  a  solutions  from  SSI  and  SS2  are  both 
large  and  near  to  1,  meaning  only  small  amount  of  uncertainty  need  to  be  reduced  in  xi 
and  X3;  iii)  The  variations  in  SSI’s  outputs  are  much  larger  than  those  in  SS2’s  outputs, 
making  it  clear  that  the  uncertainty  in  SSI  has  a  much  greater  effect  on  the  variation  in 
the  system  level  outputs  and  since  the  variation  in  SS2’s  outputs  is  always  much  less  than 
1,  the  AOVR  (±5)  for  outputs  in  SS2  might  be  overestimated;  iv)  In  both  SSI  and  SS2, 
the  variation  in  the  couplings  (compared  to  their  outputs)  contributes  significantly  to  each 
sub-system  output  variations;  however,  r/c0  in  SS2  is  much  smaller  than  1  while  most  JJC[ 

values  are  equal  to  1  (i.e.,  the  consistency  constraint  is  active  in  SSI),  meaning  that  the 
uncertainty  propagated  from  SS2  to  SSI  through  the  coupling  variable  h  has  a  greater 
effect  than  the  variation  propagated  to  SS2  in  t\.  All  those  observations  can  help  the 
designers  revise  the  settings  of  AOVR  and  TR  of  the  target  variable  and  provide  insights 
to  understanding  the  uncertainty  effects  on  this  bi-level  multi-disciplinary  problem. 

3.4.2  Engineering  Example 

To  further  demonstrate  the  capabilities  of  MIMOSA  approach,  a  battery-powered 
right  angle  grinder  model  was  chosen  for  its  multi-disciplinary  nature  and  its  combination 
of  continuous  and  discrete  parameters.  This  angle  grinder  model  was  developed  based  on 
an  all-at-once  model  first  presented  by  Williams  et  al.,  [2008].  Here,  Williams’  model  is 
decomposed  into  two  coupled  sub-system  and  a  new  battery  sub-system  is  added, 
producing  a  bi-level  multi-disciplinary  system  with  three  fully  coupled  sub-systems. 
Figure  3.10  depicts  a  graphical  overview  of  this  new  angle  grinder  model.  The  grinder 
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system  analysis  models  are  collections  of  closed  form  equations,  look-up  tables  and 
conditional  functions  used  to  described  the  various  performance  and  constraint  functions 
of  the  system  as  functions  of  geometric  and  performance  parameters.  Although  the 
grinder  model  contains  no  complex  analyses,  such  as  finite  element  analysis,  the  system 
and  sub-system  analysis  models  are  treated  as  “black-box”  functions  considering  only 
inputs  and  outputs,  which  mirrors  many  engineering  analysis  models.  The  inclusion  of 
more  complex  analysis  examples  would  not  change  the  MIMOSA  approach  beyond  an 
obvious  increase  in  the  required  computational  effort. 


^5^01  SSI:  battery  pack 

Figure  3.10:  Engineering  Example:  Right  Angle  Grinder 


The  overall  grinder  system  model  (SSO)  consists  of  three  sub-systems  which  define 
the  design  of  the  three  main  physical  components  of  the  power  tool:  a  battery  pack  model 
(SSI),  an  electric  motor  model  (SS2)  and  a  bevel  gear  assembly  (SS3).  The  design  of 
each  of  the  three  sub-systems,  as  shown  in  Figure  3.11,  is  accomplished  through  the  use 
of  various  equations  that  describe  the  physics,  performance  and  geometry  of  the  specific 
sub-system.  Additionally,  each  of  the  grinder’s  three  sub-systems  is  connected  to  other 
sub-systems  through  coupling  variables.  For  these  reasons,  this  model  provides  a  good 
platform  for  demonstrating  the  MIMOSA  approach.  Each  of  the  sub-systems  is  briefly 
described  in  the  following  paragraphs. 
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Shared  system  inputs: 

Gearratio  (Gr) 

Motor  shaft  diameter  (SD) 
Number  of  battery  cells  (nb) 
Nominal  cell  voltage  ( CellVots ) 
Density  of  steel  (pslt€l)  d 


4 


Grinder  mass 
Grinder  power 


Figure  3.11:  Design  Variables,  Couplings,  and  Outputs  in  Grinder  Example 


As  shown  in  Figure  3.1 1,  at  the  system  level,  there  are  three  design  variables  that 
are  used  by  more  than  one  sub-system.  These  are  gear  ratio  ( Gr ),  motor  shaft  diameter 
(So),  and  number  of  battery  cells  in  the  battery  pack  («*,),  in  which  nb  is  an  integer 
variable.  The  overall  system  design  outputs  of  interest  are  the  total  grinder  mass  and  the 
total  grinder  power.  Each  sub-system  has  its  own  design  variables  and  local  outputs,  as 
shown  in  Figure  3.1 1.  More  importantly,  there  are  three  coupling  variables  that  connect 
three  sub-systems:  current  (7)  from  SSI  to  SS2,  torque  load  from  SS2  to  SS3  and  shaft 
mass  from  SS3  to  SS2.  In  order  to  decompose  the  system  into  a  decomposed  framework, 
three  target  variables  are  introduced  in  SSO,  one  for  each  coupling.  Thus,  SSO  has  six 
design  variables  in  total.  Two  SSO’s  parameters  have  interval  uncertainty  in  this  example, 
as  shown  in  Table  3.4.  The  design  variables,  parameters,  and  outputs  of  SSO  and  each 
sub-system  are  detailed  in  the  first  column  to  the  third  column,  respectively,  in  Table  3.4 
to  Table  3.7,  where  a  bold  font  is  used  to  highlight  the  design  variables  or  parameters 
which  have  interval  uncertainty  in  SSO  and  each  sub-system.  As  discussed  earlier  in 
Section  3.3,  the  target  variables  must  have  an  original  tolerance  region  and  a  retained 
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tolerance  region  controlled  by  their  a  vector  values.  Additionally  the  outputs  of  interest  in 


each  sub-system  have  also  been  bolded  and  will  necessarily  have  resulting  variation. 

Each  sub-system  has  its  own  design  variables  and  parameters.  For  this  study  two 
parameters  or  design  variables  for  each  sub-system  are  assumed  to  have  interval 
uncertainty,  so  there  are  two  au  values  established  in  each  sub-system  to  control  the 
possible  uncertainty  level  reduction.  Each  sub-system  also  has  two  outputs  of  interest, 
selected  from  their  analysis  outputs,  except  SS3  (Bevel  Gear)  which  has  only  one  output 
due  to  the  relatively  simple  nature  of  the  model.  Table  3.4  through  Table  3.7  outlines  the 
specifics  of  the  4  models  that  comprise  the  right  angle  grinder  system. 


Table  3.4:  Grinder  System  (SSO) 


Grinder  Variables: 

Grinder  Parameters: 

Grinder  Outputs  (Units): 

Number  of  battery  cells  ( nb ) 

External  components  mass  (mext) 

Grinder  mass  (lbs) 

Gear  ratio  (Gr) 

Cell  voltage  (Cellv„i,,) 

Grinder  RPMs  (rpm) 

Motor  shaft  diameter  (SD) 

Density  of  steel  (psteei) 

Grinder  current  (A) 

Target  Torque  load 

Grinder  power  (W) 

Target  Shaft  Mass 

Grinder  duration  (hr) 

Target  Current 

Grinder  grirth  (in) 

Table  3.5:  Battery  Sub-System  (SSI) 


Battery  Variables: 

Battery  Parameters: 

Battery  Outputs  (Units): 

Battery  cell  height  (hceii) 

NiCd  cell  voltage  (Vce„) 

Battery  cell  mass  (lbs) 

Ni  reactant  sheet  thickness  (G,) 

Density  of  Nickel  (pNi) 

Battery  cell  capacity  (Ah) 

Cd  reactant  sheet  thickness  (tcd) 

Density  of  Cadmium)/)  Cd) 

Grinder  voltage  (V) 

Separator  sheet  thickness  (tSep) 

Density  of  Separator  (pseP) 

Battery  pack  mass  (lbs) 

cell  discharge  timc(7)/) 

Density  of  cell  wall  material  (pcw) 

Battery  cell  coil  turns  (Nb)  -  integer 

Battery  cell  wall  thickness  (cwt) 
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Table  3.6:  Electric  Motor  Sub-System  (SS2) 


Motor  Variables 

Motor  Parameters 

Motor  Outputs  (Units) 

Motor  armature  wire  turns  (Nc) 

Density  of  steel  (psteej) 

Motor  mass  (lbs) 

Stator  armature  wire  turns  (Ns) 

Shear  strength  of  steel  (zsteei) 

Motor  girth  (in) 

Stator  outer  radius  (R„) 

Demsity  of  copper  ( pCu ) 

Motor  RPM  (rpm) 

Stator  thickness  ( ts ) 

20  awg  wire  resistivity  (Rw) 

Torque  load  (ft-lbs) 

Gap  length  (LGaB) 

20  awg  wire  cross-section  area  (Aw) 

Stack  length  (LStack) 

Brush  loss  factor  ( aBrUsh ) 

Grinder  RPMs  -  loaded  ( RPMLoad ) 

Permeability  of  free  space  (jli0 ) 

Permeability  of  steel  ( justeei ) 

Table  3.7:  Bevel  Gear  Sub-System  (SS3) 


Gear  Variables: 

Gear  Parameters: 

Gear  Outputs  (Units): 

Motor  shaft  length  (SL) 

Pressure  angle  ( 6P ) 

Gear  mass  (lbs) 

Pinion  pitch  diameter  ( D„ ) 

Face  width  (Wf) 

Gear  elasticity  factor  (Ze) 

Amplification  factor  (Ka) 

Load  distribution  factor  (Km) 

Geometric  factor  (Gh) 

Density  of  steel  (psteed 

Arbor  Mass  (mA) 

Commutator  Mass  ( mc ) 

Pinion  Shaft  Mass  (/%) 

In  this  engineering  example,  it  is  assumed  that  the  designer  does  not  have  a 


specified  AOVR  for  SSO  or  sub-systems  and  would  like  to  understand  the  effects  of  input 
uncertainty  on  any  possible  variation  ranges  for  the  outputs  at  both  system  and  sub¬ 
system  levels.  In  this  regard  all  output  variations  are  normalized  to  their  nominal  output 
values  and  the  MIMOSA  formulation  in  Eqn.  (3.9)  is  used  for  each  sub-system.  A  grinder 
design  was  obtained  to  serve  as  the  nominal  or  candidate  solution  using  the  the 
collaborative  optimization  approach  of  Aute  and  Azarm  [2006].  Using  this  candidate 
design,  a  set  of  MIMOSA  Pareto  solutions  is  obtained  using  Eqn.  (3.6)  and  Eqn.  (3.9)  at 
the  system  and  sub-system  levels  with  three  typical  solutions  as  shown  in  Figure  3.12. 
Different  from  the  numerical  example,  the  optimal  a,  solution  with  the  minimum  Rssi  is 
selected  from  SSz’s  Pareto  in  this  engineering  example  and  used  to  calculate  the  outputs 
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of  the  SA  problem  in  SSO.  The  Rsso  is  shown  as  a  percentage  of  the  nominal  output 
function  values,  a  =  { a\,  a-i,  oc$,  CC\,  0C5}  is  the  uncertainty  reduction  vector  corresponding 
for  {target  current,  target  torque  load,  target  shaft  mass,  cell  volts,  density  of  steel),  as 
shown  in  bold  in  Table  3.4.  The  trade-off  between  Investment  and  the  variation  in  SSO’s 
outputs  Rsso  are  clearly  shown  in  Figure  3.12.  Figure  3.13  shows  the  correlation  plot 
figures  for  five  a  vector  elements  along  with  Rsso  and  Investmentsso  for  all  Pareto  a 
solutions  shown  in  Figure  3.12.  The  correlation  coefficient  values  for  all  Pareto  a 
solutions  in  Figure  3. 12  are  also  given  in  Table  3.8. 


(a) 


(b) 


1 


0.8 


0.2 


0 


(c)  (d) 

Figure  3.12:  Grinder  Design:  Pareto  Solutions  for  System  and  Sub-System  Levels: 

(a)  SSO,  (b)  SSI,  (c)  SS2,  and  (d)  SS3 
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As  shown  in  Figure  3.12(a),  with  the  obtained  largest  uncertainty  intervals  (am  = 


{0.95  1.00  0.98  0.68  0.98}),  the  variations  in  SSO’s  outputs  are  about  6.2%  of  their 
nominal  values.  With  more  uncertainty  being  reduced  (i.e.,  a\  =  {0.55  0.87  0.98  0.01 
0.28}),  the  Rsso  can  be  reduced  to  around  0.3%  of  the  nominal  value. 
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Figure  3.13:  Grinder  Design:  Plots  of  Correlations  among  «i  Investment  and  R 

As  clearly  shown  in  Figure  3.13  and  Table  3.8  (in  the  grey  cells),  among  those  five 
uncertain  parameters,  Cellvohs  (04)  has  the  strongest,  while  target  shaft  mass  has  the 
weakest,  correlation  to  the  system  output  variation.  Other  three  uncertain  parameters  are 
in  the  middle.  In  this  regard,  more  investment  should  be  used  to  reduce  the  uncertainty  in 
the  cell  volts  and  the  uncertainty  in  the  current  for  the  battery  part  (the  second  strongest 
correlated  to  /?ssoX  which  is  consistent  with  expectations  since  the  voltage  and  output 
current  of  the  battery  can  significantly  affect  the  performance  of  the  grinder. 
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Table  3.8:  Correlation  Coefficient  Matrix  of  Pareto  Solutions  at  System  Level 


Correlation 

Coefficient 

ctri 

a2 

C(i 

at 

as 

R 

Investment 

ctri 

1 

- 

- 

- 

- 

- 

- 

a2 

0.75 

1 

- 

- 

- 

- 

- 

OCi 

-0.18 

-0.05 

1 

- 

- 

- 

- 

«4 

0.31 

0.23 

0.25 

1 

- 

- 

- 

a5 

0.82 

0.64 

-0.19 

0.29 

1 

- 

- 

R 

0.34 

0.25 

0.24 

1 

0.32 

1 

-0.98 

Investment 

-0.51 

-0.39 

-0.19 

-0.97 

-0.50 

-0.98 

1 

For  the  sake  of  brevity  detailed  solutions  from  each  of  the  sub-systems  are  not 
given  here.  However,  some  additional  observations  are  made  in  this  example.  Since  a 
single  optimal  solution  that  possesses  the  minimum  Rssi  from  each  sub-system’s  Pareto  is 
selected  for  use  in  the  corresponding  SSO  solution,  the  Investment  required  by  all  three 
sub-systems  are  relatively  large,  as  shown  in  Figure  3.12(b)  to  3.12(d).  It  is  also  observed 
that  although  all  three  sub-systems  have  comparable  variations  in  their  mass  values,  after 
normalizing  them  by  their  nominal  values,  the  normalized  variation  in  SS2’s  outputs 
(Figure  3.12(c))  is  considerably  larger  than  the  variation  in  SSI  and  SS3  (Figure  3.12(b) 
and  3. 12(d)).  This  shows  that  SS2  is  more  sensitive  to  the  input  uncertainty  in  this  grinder 
model.  The  variations  in  the  coupling  variables  are  always  less  than  the  RTR  of  the  target 
variables  due  to  the  consistency  constraint.  Moreover,  at  the  sub-system  level,  three 
uncertain  parameters,  the  nickel  reactant  sheet  thickness  in  SSI,  the  stator  outer  radius  in 
SS2,  and  the  pinion  pitch  diameter  in  SS3,  compared  to  another  uncertainty  parameter  in 
the  corresponding  sub-system,  are  more  strongly  correlated  to  the  output  variations  in 
their  sub-systems.  They  have  been  underlined  through  Table  3.3  to  Table  3.6.  More 
investment  should  be  spent  on  reducing  uncertainty  in  those  three  parameters.  This  is  an 
interesting  finding  recalling  that  in  the  current  algorithm  from  the  sub-system  level  the 
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uncertainty  reduction  vector  (a)  values  are  selected  in  order  to  produce  the  greatest 
reduction  in  sub-system  output  variation  (minimum  R  without  regard  for  Investment).  In 
the  case  of  the  battery  sub-system,  this  choice  clearly  demonstrates  that  controlling  the 
nickel  reactant  sheet  thickness  parameter  is  far  more  important  than  controlling  the 
density  of  the  nickel  used  to  ensure  the  performance  of  the  battery  sub-system.  Figure 
3.14  is  a  plot  of  the  corresponding  «  values  for  tm  and  pm  from  SSI  for  all  Pareto 
solutions  at  the  system  level  shown  in  Figure  3.12.  As  can  be  seen  on  the  figure,  the 
uncertainty  reduction  value  (a*)  for  tm  is  much  smaller  than  the  corresponding  uncertainty 
reduction  value  for  pm,  which  indicates  that  in  order  to  produce  optimal  variation 
reduction  in  the  outputs  for  the  battery  sub-system,  the  uncertainty  in  the  geometric 
parameter  must  be  much  more  tightly  controlled. 


Figure  3.14:  a  Values  of  tNi  and  pNi  for  Pareto  Solutions  at  System  Level 

Based  on  the  data  presented,  in  order  to  optimally  reduce  the  variation  in  the 
outputs  at  the  system  and  sub-system  levels,  the  designer  of  this  right  angle  grinder 
should  reduce  the  uncertainty  mainly  in  the  four  parameters  (or  design  variables)  which 
are  underlined  through  Table  3.4  to  Table  3.7.  The  results  shown  here,  while  clearly 
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based  on  the  grinder  model  used  for  this  example,  highlight  the  greater  capability  of  the 
MIMOSA  approach  to  efficiently  identify  the  most  important  uncertain  parameters  in  a 
complex  model  during  the  conceptual  phase  of  design.  This  discussion  is  merely  an 
example  of  the  type  of  data  that  could  be  extracted  from  each  of  the  other  sub-systems 
and  how  it  might  be  analyzed. 

3.5  SUMMARY  OF  RESEARCH  THRUST  1 

In  this  chapter  a  new  global  SA  method,  called  MIMOSA,  is  presented.  MIMOSA 
has  the  ability  to  analyze  the  effects  of  uncertainty  on  system  and  sub-system 
performance  for  fully  coupled  multi-output  multi-disciplinary  problems  in  which 
uncertainty  exists  not  only  in  parameters  in  each  sub-system  but  also  in  the  sub-system 
couplings.  Given  multiple  interval  uncertainties  for  parameters  at  the  system  and  sub¬ 
system  levels,  MIMOSA  can  optimally  identify  how  much  the  investment  is  necessary  to 
optimally  reduce  uncertainty  in  input  parameters  in  order  to  ensure  a  desired  level  of 
variation  in  the  system  and  sub-systems’  output  values,  considering  a  single  design  or  a 
set  of  designs.  Two  metrics,  R  and  Investment,  have  been  used  in  the  SA  problems  in  SSO 
and  sub-systems  to  quantify  the  variation  in  multiple  system  and/or  sub-system  outputs 
and  the  cost  used  to  reduce  the  input  uncertainty.  MIMOSA  can  also  efficiently 
determine  the  relative  importance  among  uncertain  parameters  in  the  system  and  sub¬ 
systems  in  terms  of  uncertainty  reduction.  Sub-systems’  relative  importance,  in  terms  of 
their  contribution  to  the  entire  system  performance  variation,  can  also  be  analyzed  using 
this  new  approach.  MIMOSA  is  a  global  SA  approach  which  uses  interval  uncertainty  to 
quantify  the  uncertainty  instead  of  gradient  information  or  probability  density  functions. 
Because  MIMOSA  is  accomplished  by  using  the  MOGA  as  its  optimization  solver,  it  is 
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capable  of  handling  both  continuous  and  discrete  design  variables  and/or  parameters. 
Finally,  MIMOSA  is  flexible  enough  to  be  able  to  handle  various  designer  preferences, 
including  the  limits  on  parameter  uncertainty  reduction  levels,  different  acceptable  output 
variation  levels,  and  different  selection  strategies  if  applicable. 

The  applicability  and  capabilities  of  the  MIMOSA  approach  have  been 
demonstrated  using  a  numerical  and  an  engineering  example,  both  of  which  have 
multiple  fully  coupled  sub-systems.  The  numerical  example  had  two  outputs  in  each  of 
two  sub-systems  while  the  engineering  example  includes  three  sub-systems  and  has 
mixed  continuous-discrete  design  variables.  From  the  results  obtained  in  these  two 
examples,  the  trade-off  between  the  investment  used  to  reduce  the  uncertainty  levels  in 
the  input  parameters  and  the  variations  in  the  system  outputs  are  clearly  demonstrated. 
Both  examples  show  the  ability  of  MIMOSA  to  optimally  determine  the  best  uses  of 
limited  investment  available  for  uncertainty  reduction  in  the  attempt  to  improve  system 
performance  in  terms  of  reducing  output  variations.  The  correlation  between  uncertain 
parameters  and  the  variations  in  the  system  and  sub-system  outputs  are  easily  determined 
based  on  the  obtained  Pareto  solutions  for  R  vs.  Investment.  Both  examples  specifically 
present  how  the  MIMOSA  approach  can  identify  the  sub-system  in  a  decomposed  multi¬ 
sub-system  problem  that  is  the  most  sensitive  to  the  uncertainty  and  has  the  greatest 
effect  on  the  system  level  performance  under  uncertainty.  The  engineering  example  also 
demonstrates  how  MIMOSA  is  capable  of  isolating  parameters  whose  uncertainty  greatly 
influence  system  and  sub-system  outputs  in  contrast  to  those  who  do  not.  The  importance 
of  parameters  in  each  of  the  sub-systems  of  the  angle  grinder  has  been  identified  as  well. 
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The  MIMOSA  approach  presented  is  adaptable  to  many  different  types  of 
engineering  problems  and  thus  has  a  wide  range  of  applications  not  specifically  addressed 
in  this  research  thrust.  As  a  result  of  this  flexibility,  the  MIMOSA  approach  is  an 
excellent  tool  for  clearly  understanding  and  optimally  eliminating  the  wide-spread  effects 
of  uncertainty  in  many  different  engineering  design  efforts  based  on  designer  preferences, 
limitations  and/or  goals.  For  instance,  Investment  could  be  further  defined  to  include  a 
real  cost  function  associated  with  specific  uncertainty  reductions,  and  that  cost  function 
needs  not  be  the  same  for  all  uncertain  parameters  within  a  problem  (e.g.,  [Acar  et  al., 
2007]  and  [Kale  and  Haftka,  2008]).  MIMOSA  could  also  be  used  to  ignore  (redundant) 
design  parameters  or  outputs  from  the  design  process  that  are  unimportant  or  already 
insensitive  to  uncertainty  in  favor  of  using  valuable  computational  resources  for  the  more 
important  or  more  critical  design  aspects,  thus  simplifying  a  complicated  problem  down 
to  a  more  manageable  size  consisting  of  more  relevant  issues. 

The  MIMOSA  approach  is  clearly  quite  capable  and  very  useful  given  the  correct 
system  properties,  such  as  a  preexisting  candidate  design  and  input  uncertainty  that  is 
reducible  about  known  nominal  parameter  values.  However,  as  stated  in  Chapter  1, 
MIMOSA  does  not  consider  engineering  feasibility  which  is  a  critical  element  of  many 
engineering  design  challenges.  The  work  presented  in  the  next  chapter  attempts  to 
address  this  shortcoming. 
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CHAPTER  4:  MULTI-DISCIPLINARY  COMBINED 
SENSITIVITY  ANALYSIS  (MICOSA) 

4.1  INTRODUCTION  TO  RESEARCH  THRUST  2 

The  goal  of  the  second  research  thrust  was  quite  simply  to  extend  the  MIMOSA 
approach  of  Chapter  3  to  multi-disciplinary  systems  that  may  include  infeasibility  under 
uncertainty.  The  MIMOSA  algorithm  presented  in  Chapter  3  provides  a  multi¬ 
disciplinary  SA  framework,  but  it  does  not  acknowledge  the  possibility  for  infeasibility 
due  to  the  input  parameter  uncertainty  in  an  engineering  system.  Just  because  a  system’s 
objectives,  for  example,  are  insensitive  to  the  variation  in  its  input  parameters  does  not 
mean  that  the  feasibility  of  the  system  is  necessarily  ensured  under  uncertainty. 
Furthermore  it  may  at  times  be  more  important  for  the  feasibility  of  a  design  to  be 
insensitive  to  uncertainty  than  for  the  design  be  optimal  in  an  objective  sense.  The  ability 
to  use  uncertainty  reduction  to  improve  the  performance  of  a  system  under  uncertainty  is 
a  powerful  tool.  However,  there  is  a  very  real  possibility  for  uncertainty  to  lead  to  failure 
(infeasibility)  at  either  the  system  level  or  sub-system  level  of  a  multi-disciplinary 
system.  The  work  presented  in  this  is  chapter  is  a  simple  but  necessary  modification  to 
the  MIMOSA  approach  presented  in  previous  chapter  which  makes  it  possible  to  use 
uncertainty  reduction  to  also  ensure  engineering  feasibility  in  addition  to  producing 
reduced  output  function  variations.  This  new  approach  is  called  Multi-disciplinary, 
multi-objective  COmbined  Sensitivity  Analysis  (MICOSA);  with  the  word  “combined” 
chosen  to  indicate  that  this  approach  considers  the  effects  of  both  objective  and  constraint 
function  variations  in  a  combined  fashion. 
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4.2  BACKGROUND  AND  TERMINOLOGY 


This  section  contains  some  relevant  background  and  terminology  used  throughout 
the  rest  of  this  chapter. 

4.2.1  MIMOSA  Review 

As  previously  stated,  the  MICOSA  approach  is  a  direct  extension  of  the  MIMOSA 
approach  presented  in  Chapter  3.  The  MICOSA  approach  uses  the  same  framework  and 
is  based  on  the  same  assumptions  as  the  MIMOSA  approach.  All  of  the  relevant  details  of 
the  MIMOSA  approach  can  be  found  in  the  preceding  chapter. 

4.2.2  Objective  and  Constraint  Function  Variation  Metrics 

In  the  previous  chapter,  the  output  functions  analyzed  by  MIMOSA  could  be 
either  objectives,  constraints,  or  both,  and  the  formulation  of  the  approach  made  no 
distinction.  As  a  result,  in  order  to  extend  MIMOSA  to  also  consider  engineering 
feasibility  a  distinction  between  objectives  and  constraints  is  necessary.  Eqn.  (4.1)  adapts 
the  formulation  for  R  presented  in  Chapter  3  for  objective  function(s)  only,  while  Eqn. 
(4.2)  adapts  the  R  formulation  for  constraint  function(s)  only.  Recall  that  the  R  metric 
presented  in  the  previous  chapter  is  a  function  of  the  a  vector,  which  is  a  measure  of 
uncertainty  reduction  for  the  K  input  parameters  to  a  system.  In  Eqns.  (4.1)  and  (4.2)  the 
vector  pv  is  established,  which  represents  a  single  realization  of  the  uncertain  values  p 
bounded  by  the  retained  uncertain  interval  [po  -  «  °  A p,  p»  +  a  A p].  This  vector  will  be 
used  throughout  this  chapter  (and  later  on  in  the  dissertation)  as  a  decision  variable  when 
attempting  to  quantify  the  effect  of  input  uncertainty. 
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It  should  be  noted  that  the  value  for  Rg  will  be  greater  than  1  for  an  infeasible  level 
of  input  uncertainty  based  on  the  formulation  of  Eqn.  (4.2).  This  capability  is  important 
because  it  makes  it  possible  to  easily  detect  when  a  particular  combination  of  input 
uncertainties  will  lead  to  potential  engineering  infeasibility. 


Pr 


Figure  4.1:  Effects  of  Reduced  Uncertainty  on  Objectives  and  Constraints 

Figure  4. 1  depicts  Rf  and  Rg  graphically  given  input  parameter  uncertainty.  Note 
that  uncertainty  in  p  will  lead  to  uncertainty  in  /  and  g,  as  discussed  in  Chapter  2.  Note 
that  for  the  light  grey  (larger)  uncertain  region,  the  value  for  gj  is  greater  than  0  indicating 
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infeasibility,  while  the  darker  grey  uncertain  region  ensures  that  all  possible  outcomes  of 
p  will  result  in  all  constraint  functions  (gi  and  g2)  being  less  than  0  and  thus  feasible. 
Refer  to  the  discussion  on  multi-objective  optimization  in  Chapter  2  for  details  on  the 
violation  of  constraint  functions  within  the  context  of  optimal  design. 

4.2.3  Combined  Output  Variation  Metric 

To  simultaneously  consider  both  objective  and  constraint  function  variations,  the 
following  new  metric,  called  Rc,  is  presented.  This  metric  simply  compares  the  resulting 
Rf  and  Rg  values  for  a  given  level  of  input  uncertainty  and  select  the  larger  value.  This 
formulation  makes  it  possible  for  Rc  to  reflect  the  system  performance  measure,  either 
objective  function  variation(s)  or  constraint  function  variation(s),  that  is  more  important 
for  a  given  level  of  input  uncertainty  as  specified  by  a.  This  metric  will  be  used  as  an 
objective  function  in  the  MICOSA  formulation  which  will  be  presented  later  in  this 
chapter. 

Rc  =max  (Rf,Rg)  (4.3) 

4.3  MICOSA  FORMULATION 

In  order  to  account  for  the  shortcoming  of  the  MIMOSA  approach  previously 
discussed,  the  following  extension  was  developed.  This  extension,  called  MICOSA, 
seeks  to  use  uncertainty  reduction  as  a  means  for  improving  the  objective  performance  of 
a  multi-disciplinary  system  design,  while  simultaneously  ensuring  the  feasibility  of  the 
system  under  uncertainty.  The  MICOSA  algorithm  includes  two  changes  from  the 
MIMOSA  formulation  presented  in  the  last  chapter.  First,  the  first  objective  function  of 
the  MICOSA  formulation  is  changed  to  minimize  Rc,  vice  R  as  was  used  in  the  previous 
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chapter.  Recall  that  Rc  is  a  function  of  R/  and  Rg,  which  are  given  in  Eqns.  (4.1)  and  (4.2) 
respectively.  Using  Rc  the  MICOSA  algorithm  seeks  to  reduce  variations  in  either  the 
objective  or  constraint  function(s)  at  the  system  and  sub-system  levels,  whichever  are 
greater  given  the  current  levels  of  input  uncertainty.  The  second  change  is  the  inclusion 
of  a  constraint  that  Rg  at  both  the  system  and  sub-system  levels  must  be  less  than  or  equal 
to  1.  Using  the  formation  for  Rg  provided  in  Eqn.  (4.2),  a  value  of  Rg  less  than  or  equal 
to  1  will  indicate  a  feasible  system  as  previously  discussed,  assuming  the  nominal 
(deterministic)  design  is  feasible.  Eqn.  (4.4a)  provides  this  new  MICOSA  formulation, 
which  is  applicable  to  a  single  disciplinary  system  or  any  sub-system  (SSi)  within  a 
multi-disciplinary  framework.  If  Eqn.  (4.4a)  is  used  to  analyze  a  single  disciplinary 
system,  the  name  of  the  approach  is  shorted  to  Combined  Sensitivity  Analysis  (COSA) 
for  simplicity.  In  this  chapter  the  Investment  metric  has  been  abbreviated  with  the  letter  I 
only,  but  the  formulation  used  is  identical  to  that  provided  for  Investment  in  Chapter  3. 

min  Rc,SSi 

min  I ss. 

s.t.  RgSSi<  1  (4-4a) 

where  RCSSi  =  ma x(RfSSi ,  RgSSi ) 

For  demonstration  purposes,  an  unconstrained  formulation  of  the  COSA 
algorithm  is  provided  in  Eqn.  (4.4b).  Notice  that  Eqn.  (4.4b)  is  identical  to  Eqn.  (4.4a) 
except  for  the  lack  of  the  constraint  on  the  value  of  Rg.  This  formulation  is  provided  for 
comparison  purposes  and  will  be  used  to  show  that  the  formulation  in  Eqn.  (4.4a)  is 
necessary  to  achieve  the  stated  goals  of  this  research  thrust,  namely  to  ensure  feasibility 
of  the  engineering  system  under  uncertainty. 
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(4.4b) 


min  RCSSi 

m'n  /  SSl 

where  RCSSi  =  ma \(RfSSi,  RgSSi) 

MICOSA  requires  simply  employing  the  MIMOSA  framework  with  the  new 
COSA  formulation  at  the  system  and  sub-system  levels,  as  shown  in  Figure  4.2.  The 
steps  and  procedures  for  employing  the  MICOSA  approach  are  identical  to  that  of 
MIMOSA  as  presented  in  the  previous  chapter.  The  only  changes  are  those  detailed 
above  with  regard  to  the  new  objective  function  Rc  and  the  inclusion  of  the  Rf,  constraint 
at  the  system  and  sub-system  levels. 


Figure  4.2:  MICOSA  Framework 


4.4  EXAMPLES  AND  RESULTS 

In  an  effort  to  demonstrate  the  capabilities  of  the  approach,  the  MICOSA  approach 
was  applied  to  two  examples,  a  single  disciplinary  propulsor  model  and  a  multi¬ 
disciplinary  unmanned  underwater  vehicle  (UUV)  model.  The  single  disciplinary  model 
demonstrates  the  ability  of  the  approach  to  ensure  the  feasibility  of  the  system  through 
uncertainty  reduction.  The  multi-disciplinary  example  then  demonstrates  the  ability  of 
the  approach  to  handle  more  complex  system  models  within  a  decomposed  framework. 
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4.4.1  Single  Disciplinary  Propulsor  Example 


To  verify  the  COSA  algorithm,  the  approach  was  used  to  analyze  a  candidate 
design  for  an  undersea  vehicle  propulsor.  The  propulsor  system  analysis  model  is  a  set  of 
meta-models  fitted  to  a  set  of  CFD  data  for  a  notional  propulsor  provided  to  the 
University  of  Maryland  by  the  Naval  Surface  Warfare  Center  as  part  of  active  research 
collaborations.  Figure  4.3  depicts  this  example  problem.  This  figure  contains  several 
pieces  of  information  required  for  employing  COSA,  including:  the  underlying 
optimization  problem  (e.g.  maximizing  propulsor  efficiency  and  minimizing  propulsor 
noise),  the  parameter  values  for  the  candidate  design  of  interest  (e.g.  OD  =  8. 1 1  inches) 
and  the  nominal  uncertainty  levels  (e.g.  1%  of  nominal  for  the  outer  diameter  parameter 
value  of  8. 1 1  inches)  for  each  of  the  input  parameters. 


Figure  4.3:  UUV  Propulsor  Model 


Using  the  single  disciplinary  COSA  algorithm  the  above  problem  was  solved 
twice  and  the  results  are  presented  in  Figure  4.4,  once  using  the  constrained  formulation 
provided  in  Eqn.  (4.4a)  and  once  using  the  unconstrained  formulation  provided  in  Eqn. 
(4.4b).  The  Pareto  frontiers  provided  on  the  left  side  of  Figure  4.4  show  the  expected 
tradeoff  between  Rc  and  /.  Of  particular  interest  is  the  demonstrated  need  for  the  Rg 
constraint.  As  seen  in  the  two  solutions  highlighted  by  the  dashed  circle  on  both  sides  of 
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the  figure,  the  Rc  and  /  values  for  the  constrained  and  unconstrained  solutions  are  similar 
(left  side  figure),  but  upon  further  examination  the  constrained  solution  is  feasible  (Rg  < 
1)  and  the  unconstrained  solution  is  infeasible  ( Rx  >  1)  as  seen  on  the  right  side  plot, 
which  is  not  a  Pareto  frontier  but  rather  a  plot  of  R x  vs  /  for  comparison  purposes. 


Figure  4.4:  Combined  Sensitivity  Analysis  Results  for  Propulsor  Design 


These  two  solutions  differ  in  terms  of  how  the  reduced  variation  ( Rc )  is  produced. 
Figure  4.5  depicts  the  uncertainty  bands  for  each  of  the  four  parameters  considered  in  the 
example,  comparing  the  nominal  uncertainty  (black  bands  on  the  left  side  of  each  plot), 
the  retained  uncertainty  in  the  unconstrained  case  (hollow  grey  bands  in  the  middle  of 
each  plot)  and  the  retained  uncertainty  in  the  constrained  formulation  (hollow  black 
bands  on  the  right  of  each  plot),  which  is  the  actual  COSA  approach.  In  the 
unconstrained  case  the  uncertainty  in  the  housing  mass  is  greatly  reduced  because  this 
reduces  the  variation  the  propulsor  mass  (one  of  the  objective  functions  for  the  original 
problem)  more  significantly.  However,  the  uncertainty  in  the  RPM  value  must  have  a 
greater  effect  on  feasibility  for  the  propulsor  system  and  thus,  when  feasibility  is 
considered  in  the  constrained  approach,  the  uncertainty  in  the  RPM  is  reduced  further 
while  the  housing  mass  parameter  retains  more  of  its  original  uncertainty.  Table  4.1 
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contains  the  numerical  a  values  for  the  four  parameters  discussed  above  (and  shown  in 


Figure  4.5)  for  completeness. 


OD 
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Figure  4.5:  Comparison  of  Highlighted  COSA  Solutions 
Table  4.1:  Uncertainty  Reduction  Vector  Values  for  Figure  4.5  Solutions 


a  values: 

OD 

RPM 

CT 

Housing 

Unconstrained 

0.992 

0.800 

0.921 

0.671 

Constrained 

0.940 

0.656 

0.830 

0.927 

4.4.2  Multi-Disciplinary  UUV  Example 

The  MICOSA  approach  was  also  applied  to  a  complex  multi-disciplinary  system 
model.  The  model  chosen  was  an  unmanned  underwater  vehicle  (UUV)  model 
comprised  of  5  sub-systems.  This  model  was  developed  using  open  literature  sources 
[Hamel  et  al.  2009],  For  this  application  only  the  system  level  model  and  two  of  the  sub¬ 
system  models  were  assumed  to  possess  uncertainty.  Figure  4.6  depicts  the  analysis 
system  diagram,  the  system  and  sub-system  candidate  design  nominal  values  for  all 
uncertain  inputs,  the  uncertainty  associated  the  input  parameters  and  the  underlying 
design  optimization  problem  for  each  of  the  systems  to  be  analyzed.  Much  of  the  details 
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of  this  model  are  omitted  in  the  interest  of  brevity,  as  the  complete  system  model  consists 
of  over  a  hundred  input  parameters  and  numerous  outputs.  The  system  given  in  Figure 
4.6  was  analyzed  using  the  MICOSA  formulation  and  the  system  level  results  along  with 
a  sample  of  the  sub-system  level  results  obtained  are  provided  in  Figure  4.7.  It  should  be 
noted  that  sub-system  results  presents  are  associated  with  just  one  of  the  Pareto  solution 
in  the  system  level  solution  set.  As  with  the  MIMOSA  approach,  a  complete  set  of 
Pareto  solutions  are  produced  at  the  sub-system  level  for  each  solution  point  generated  at 
the  system  level  and  the  vast  majority  of  the  sub-system  solutions  are  not  presented  in 
this  dissertation  in  the  interest  of  brevity. 


Outer  Diameter  (shared) 
8.11  in 
Speed  (local) 

50  kts 

RPM  (shared) 

2406  RPM 

min  Mass 
max  PK 
s.t. 

Eng.  Constraints 

(ie.  max  weight) 

Target  Strength 
99  dB 

GNC*  Void  Diameter 
4.6  in 

max  Directivity  Index 
min  GNC*  Power 
s.t. 

Engineering  Constraints 

(ie.  max  length) 

Collaborative  Constraints 

(ie.  GN C* power  <  GNC*  Power  awilable) 

*  GNC  =  Guidance  and  Control 


Uncertainty  in  all  parameters  taken 
to  be  20%  of  nominal 


Payload  Length 
34.7  in 

min  Weight 
max  Psuccess 
s.t. 

Engineering  Constraints 

(ie.  max  length /diameter) 

Collaborative  Constraints 

(Le.  length  <  target  length) 


Figure  4.6:  UUV  MICOSA  Example 
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SSO  Pareto- UUV 


Rc 

Sample  SSI  Pareto  -  GNC  Sample  SS2  Pareto  -  Payload 


Rc  Rc 

Figure  4.7:  Sample  UUV  MICOSA  Results 

A  few  observations  can  be  made  from  Figure  4.7.  First,  it  is  clear  that  MICOSA 
approach  produces  results  that  are  very  similar  in  nature  to  those  provided  by  MIMOSA. 
These  results  provide  designers  with  insight  into  how  best  to  allocate  limited  resources  in 
an  effort  to  improve  the  performance  of  the  overall  system.  Furthermore,  MICOSA  also 
is  capable  of  determining  the  relative  importance  of  different  sub-systems  when 
considering  both  objective  function  variations  and  engineering  feasibility.  Note  that  the 
sample  payload  sub-system  results  show  a  wide  range  of  potential  solutions  ranging  from 
very  low  to  very  high  /  values,  indicating  that  a  designer  is  free  to  select  the  uncertainty 
reduction  levels  for  that  sub-system  based  purely  on  the  desired  Rc  value.  Conversely  for 
the  SSI,  the  Guidance  and  Control  (GNC)  sub-system,  a  minimum  I  value  of  ~  0.5  is 
required  for  all  solutions.  This  suggests  that  there  is  a  minimum  level  of  investment 
required  for  this  system  just  to  achieve  feasibility  (Rx  <  1)  under  uncertainty,  and  that  any 
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additional  investment  of  resources  (7  greater  than  ~  0.5)  can  then  result  in  reduced  output 
function  variations  (i.e.  better  performance).  These  results  suggests  that  the  MICOSA 
approach  is  capable  of  ensuring  that  resources  are  first  supplied  to  ensuring  that  the 
system  will  not  fail  prior  to  applying  any  uncertainty  reduction  resources  to  reducing 
other  output  performance  variations. 

4.5  SUMMARY  OF  RESEARCH  THRUST  2 

This  chapter  detailed  the  MICOSA  approach,  a  simple  extension  to  the  MIMOSA 
approach  presented  in  Chapter  3.  This  approach  possesses  all  the  same  capabilities  as 
MIMOSA,  with  the  added  benefit  of  also  considering  engineering  feasibility.  The 
capabilities  of  this  new  approach  were  demonstrated  through  two  example  problems  that 
highlighted  the  key  improvements  of  MICOSA  over  MIMOSA.  The  single  disciplinary 
example  was  used  to  demonstrate  the  necessity  of  the  Rg  constraint  to  the  formulation  of 
the  COSA  approach.  By  solving  the  single  disciplinary  example  problem  two  times, 
once  with  the  constraint  on  Rg  and  once  without,  the  capability  of  the  COSA  formulation 
to  ensure  feasibility  under  uncertainty  was  clearly  demonstrated.  The  multi-disciplinary 
example  problem  demonstrated  the  full  capability  of  the  MIMOSA  approach.  First  this 
example  showed  that  MICOSA  produced  results  comparable  to  those  produced  by 
MIMOSA,  which  was  to  be  expected.  Additionally,  the  second  example  problem 
demonstrated  the  value  of  using  Rc  as  the  objective  function  (as  opposed  to  7?/  or  Rg)  as  it 
forces  the  approach  to  apply  limited  uncertainty  reduction  resources  limiting  the  more 
critical  output  function  variations  (either  objectives  or  constraints)  first,  which  is  an 
intuitive  way  to  approach  the  use  of  limited  resources. 
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However,  it  may  not  always  be  possible  for  uncertainty  reduction  alone  to  ensure 
feasibility,  as  demonstrated  in  this  chapter.  Many  design  solutions  for  various  system 
may  only  be  feasible  when  uncertainty  is  completely  eliminated,  which  will  most  likely 
not  be  possible  to  accomplish  since  virtually  all  physical  systems  possess  some  level  of 
uncertainty.  Clearly  in  those  cases  the  MICOSA  approach,  while  certainly  very  capable, 
will  be  insufficient  for  mitigating  the  effect  of  input  parameter  uncertainty.  The 
following  chapter  addresses  this  shortcoming  of  the  MICOSA  approach. 
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CHAPTER  5:  DESIGN  IMPROVEMENT  BY  SENSITIVITY 

ANALYSIS  (DISA) 

5.1  INTRODUCTION  TO  RESEARCH  THRUST  3 

As  first  stated  in  Chapter  1,  much  of  the  current  research  in  engineering  design 
(including  the  first  two  research  thrusts  presented  in  Chapters  3  and  4  of  this  dissertation) 
is  driven  by  the  fact  that  all  real-world  engineering  systems  are  comprised  of  uncertain 
input  parameters  [Martin  and  Simpson,  2006;  Schueller  and  Jensen,  2008;  Moeller  and 
Beer,  2008;  Lee  and  Chen,  2009].  Understanding  and  managing  uncertainty  levels 
during  an  engineering  design  process  is  of  particular  importance  when  uncertainty  can 
lead  to  the  failure,  or  infeasibility,  of  an  engineering  system.  In  the  previous  chapter  of 
this  dissertation  the  MICOSA  approach  was  presented  which  focused  on  mitigating  the 
effects  of  uncertainty  on  a  system’s  feasibility  through  the  use  of  uncertainty  reduction. 
That  approach  was  proven  to  be  capable  of  ensuring  feasibility,  but  only  in  those  cases 
where  uncertainty  reduction  alone  was  sufficient,  which  may  not  always  be  the  case. 
This  chapter  presents  a  more  general  approach  to  that  problem  for  single  disciplinary 
engineering  systems. 

As  with  the  previous  two  chapters,  the  work  presented  in  this  chapter  focuses  on 
interval  input  uncertainty  to  engineering  system  analysis  models  and  assumes  that  the 
uncertain  intervals  are  reducible.  Furthermore  the  work  presented  in  this  chapter  has 
approach  the  problem  of  design  under  uncertainty  from  a  sensitivity  analysis  (SA)  point 
of  view,  as  opposed  to  a  robust  optimization  point  of  view,  due  to  the  fact  that  SA  is 
better  suited  to  uncertainty  reduction  decision  making.  The  motivation  and  background 
details  for  these  decisions  were  presented  in  detail  in  Chapter  1  of  this  dissertation. 
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It  should  be  noted  that  some  robust  approaches  have  been  used  successfully  to 
explore  opportunities  for  uncertainty  reduction  (as  opposed  to  simply  considering  all 
uncertainty  as  irreducible)  in  various  ways.  For  example,  there  are  RBDO  methods 
reported  in  the  literature  that  use  Bayesian  techniques  [Wang  et  al.,  2009],  methods  that 
sequentially  perform  RBDO  and  consider  uncertainty  reduction  [Qu  et  al.,  2003],  and 
methods  that  simultaneously  consider  RBDO  and  uncertainty  reduction  options  [Kale  and 
Haftka,  2008].  These  approaches  are  not  limited  to  irreducible  uncertainty  as  is 
traditionally  the  case  for  robust  approaches,  but  like  most  robust  approaches  they 
consider  many  drastically  different  potential  designs  in  an  effort  to  design  a  system  under 
uncertainty.  In  many  cases  a  designer  may  already  have  a  design  in  mind  and  may  not 
wish  to  consider  drastically  different  design  alternatives.  As  previously  discussed  in 
Chapter  1,  in  those  cases  an  SA  approach  is  often  preferable  to  a  robust  approach  due  to 
the  unique  capabilities  of  sensitivity  analysis. 

The  MOSA  approach  [Li  et  al.,  2009a]  that  was  first  presented  in  Chapter  2  and 
then  extended  in  Chapters  3  and  4,  is  an  SA  based  approach  for  uncertainty  reduction.  As 
previously  discussed,  MOSA  (and  MIMOSA  for  that  matter)  assume  that  uncertainty 
reduction  is  always  possible  and  that  any  design  under  consideration  will  always  be 
feasible  given  any  combination  of  uncertain  inputs.  Furthermore,  the  MICOSA  approach 
presented  in  Chapter  4  assumes  that  if  a  design  is  not  feasible  under  nominal  uncertainty, 
its  feasibility  can  be  guaranteed  through  uncertainty  reduction.  These  assumptions  may 
not  hold  for  a  design  that  is  on  or  near  the  limits  of  a  problem’s  feasible  domain  and  may 
become  infeasible  due  to  uncertainty,  regardless  of  how  much  uncertainty  levels  are 
reduced.  Efforts  were  made  by  Li  et  al.  to  address  this  shortcoming  by  combining  the 
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MOSA  algorithm  with  robust  optimization  techniques  in  an  effort  to  produce  design 
solutions  that  are  insensitive  to  irreducible  uncertainty  while  simultaneously  finding 
optimal  levels  of  uncertainty  reduction  for  all  reducible  uncertainty  [Li  et  ah,  2009c]. 
However,  since  this  approach  [Li  et  ah,  2009c]  is  a  robust  approach,  it  considers  many 
potential  designs  in  addition  to  finding  optimal  uncertainty  reduction  opportunities.  This 
multi-layered  strategy  adds  potentially  unnecessary  computational  expense  and  will  also 
produce  solutions  that  may  be  very  different  from  a  preferred  preexisting  design,  as 
previously  discussed.  The  work  presented  in  the  previous  chapter  attempted  to  address 
this  shortcoming  as  well,  but  only  for  those  cases  where  uncertainty  reduction  alone 
could  be  assured  of  guaranteeing  feasibility,  which  is  not  always  the  case. 

A  method  that  has  the  ability  to  improve  a  preexisting  design  and  simultaneously 
prevent  its  infeasibility  through  a  better  understanding  of  the  effects  of  uncertainty  would 
be  very  useful.  A  preexisting  design  in  this  sense  refers  to  either:  a)  a  design  that  is 
already  in  use  but  need  to  be  slightly  redesigned  (adjusted)  due  to  significant  changes  to 
input  parameter  values  or  uncertainty  levels;  or  b)  a  deterministic  design  solution 
generated  using  a  conceptual  design  procedure,  such  as  one  produced  by  a  deterministic 
design  optimization  procedure.  In  either  of  the  above  cases  the  effect  of  uncertainty  has 
to  be  accounted  for  carefully.  Typically  safety  factors  would  be  assigned  to  the  critical 
aspects  of  the  design,  which  could  necessarily  lead  to  some  adjustments  to  the  preexisting 
design’s  parameters.  It  would  be  preferable  in  many  cases  to  instead  utilize  information 
from  a  sensitivity  analysis  approach  to  produce  feasible  designs  under  uncertainty  in  a 
systematic  fashion  without  the  need  to  arbitrarily  guess  or  assume  safety  factor  values. 
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The  work  presented  in  this  research  thrust  seeks  to  provide  just  such  a  capability  by 
proposing  a  new  sensitivity  analysis  based  approach,  called  Design  Improvement  by 
Sensitivity  Analysis  (DISA).  This  new  approach  is  capable  of  first  finding  optimal 
uncertainty  reduction  options  for  a  given  design  and  then  suggesting  required  small 
changes  (or  adjustments)  to  the  original  design  to  ensure  the  feasibility  of  the  system 
when  necessary,  provided  uncertainty  reduction  alone  is  insufficient.  Small  adjustments 
are  defined  in  this  work  as  changes  to  the  nominal  values  of  any  uncertain  parameter  of 
an  engineering  system  within  the  original  uncertainty  interval  for  a  given  parameter. 

Additionally,  this  work  will  show  that,  if  desired,  the  DISA  approach  is  capable  of 
suggesting  those  required  small  adjustments  with  very  little  (if  any)  additional 
information  about  the  engineering  system.  This  is  accomplished  by  retaining  system 
information  obtained  during  an  initial  uncertainty  reduction  SA  procedure  and  then  using 
meta-modeling  [Srivastava  et  al.,  2004;  Martin  and  Simpson,  2005;  Wang  and  Shan, 
2007;  Allaire  and  Willcox,  2008;  Ju  and  Lee,  2008;  Shan  and  Wang,  2008]  to 
approximate  the  system  response  for  determining  the  required  design  adjustments.  As  a 
result,  this  new  approach  is  no  more  computationally  expensive  than  the  MOSA  approach 
[Li  et  al.,  2009a],  but  is  also  capable  of  ensuring  the  engineering  feasibility  of  a 
preexisting  design  without  the  need  for  feasibility  robust  optimization. 

The  basic  capability  of  the  DISA  approach  can  be  summarized  with  the  following 
simple  example.  Suppose  a  design  engineer  would  like  to  use  a  2  inch  diameter  pin  to  fit 
into  a  2  inch  diameter  hole  based  on  certain  requirements,  as  shown  on  the  top  half  of 
Figure  5.1,  but  knows  that  the  diameter  of  the  pin  (Do)  will  undoubtedly  possess  some 
level  of  uncertainty.  Clearly  no  amount  of  uncertainty  reduction  would  be  capable  of 
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ensuring  that  the  pin  would  always  fit  into  the  2  inch  diameter  hole  in  this  scenario. 
However,  if  the  nominal  diameter  of  the  pin  was  reduced  (or  adjusted)  to  a  value  slightly 
less  than  2  inches,  that  adjustment  coupled  with  some  small  amount  of  uncertainty 
reduction  could  then  guarantee  that  the  pin  would  always  fit  into  the  required  hole. 
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. 
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Figure  5.1:  Simple  Example 

The  DISA  approach  is  capable  of  solving  just  such  a  problem  involving  numerous 
design  variables,  multiple  design  objectives  and  many  constraints  for  a  system  possessing 
interval  uncertainty  in  an  efficient  manner.  Section  5.2  outlines  specific  background  and 
terminology  applicable  to  this  new  approach.  Section  5.3  provides  the  formulation  of  the 
approach  and  the  general  results  it  is  capable  of  producing.  Section  5.4  shows  two 
engineering  examples  of  varying  degrees  of  difficulty  to  demonstrate  the  applicability  of 
this  new  approach.  Section  5.5  contains  some  concluding  remarks. 

5.2  BACKGROUND  AND  TERMINOLOGY 

In  this  section  some  necessary  background,  terminology  and  concepts  will  be 
introduced.  Note  that  all  these  concepts  are  applicable  to  multi-dimensional  problems  but 
for  simplicity  two-dimensional  illustrations  are  used.  The  approach  presented  in  this 
chapter  is  an  extension  of  the  MIMOSA  approach  and  thus  builds  on  the  concepts 
presented  in  Chapters  3  and  4. 
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Recall  that  SA  approaches  assume  that  a  predetermined  design  already  exists  for  a 
system  of  interest.  Following  this  assumption  all  predetermined  inputs  into  an 
engineering  system  can  simply  considered  as  fixed  design  parameters  p.  Thus  hereafter 
in  this  chapter  the  vector  po,  as  with  the  previous  two  chapters,  is  used  to  denote  the 
nominal  values  for  all  inputs  into  a  system  for  which  a  solution,  or  design,  already  exists. 
Furthermore  this  design  po  will  be  called  a  candidate  design.  As  a  result,  the  nominal 
objective  and  constraint  functions  values  for  a  candidate  design  may  be  thought  of  simply 
as  functions  of  the  nominal  values  po 

The  work  presented  in  this  chapter  focuses  on  how  changes  to  the  parameters 
contained  in  the  vector  po,  both  in  terms  of  nominal  values  and  uncertainty  level,  affect 
the  performance  of  the  design  described  by  the  functions  /o  and  go.  As  with  the  work 
presented  in  the  previous  chapters,  if  the  uncertainty  associated  with  a  system’s  input 
parameters  (bounded  by  the  interval  [pa  -  Ap,po  +  A p])  are  reducible,  then  the  uncertainty 
reduction  vector  a  can  again  be  used  to  quantify  the  reduction  in  the  uncertainty  of  those 
parameters.  When  the  uncertainty  in  any  of  the  input  parameters  to  a  system  are  reduced 
by  decreasing  the  values  for  one  or  more  elements  in  a,  the  range  of  output  function 
variation  will  also  inevitably  be  changed.  This  propagation  of  uncertainty  from  input 
parameters  to  output  function  values  can  be  represented  by  a  set  of  resulting  uncertain 
objective  and  constraint  function  values: 

f,{Py)  i=l-J 
g\Pv)  j  =  h-,J 

for  all  pv 

where  p0-a  °  Ap  <  pr  <  p0  +  a  °  Ap 
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As  shown  in  Eqn.  (5.1),  uncertainty  propagation  is  a  function  of  the  nominal 
parameter  values  po,  the  original  uncertainty  intervals  Ap,  and  uncertainty  reduction 
vector  a.  In  Eqn.  (5.1)  the  vector  pv  is  again  used  (as  in  the  previous  chapter),  which 
represents  a  single  realization  of  the  uncertain  values  p  bounded  by  the  retained  uncertain 
interval  [p0  -  a  °  Ap,  p0  +  a  °  Ap],  and  will  be  used  throughout  the  chapter  as  a  decision 
variable  when  attempting  to  quantify  the  effect  of  uncertainty.  Figure  5.2  depicts  the 
impact  that  different  uncertainty  reduction  values  could  have  on  output  function 
variations  when  nominal  parameter  values  po  remain  unchanged. 


Figure  5.2:  Parameter  Uncertainty  Mapping  for  Various  orValues 


5.2.7  Design  Adjustment  Vector 

Small  adjustments  to  the  nominal  values  of  the  parameter  vector  po,  as  discussed  in 
Section  5.1,  provide  an  obvious  means  for  ensuring  the  feasibility  of  a  design  that  cannot 
become  feasible  through  uncertainty  reduction  alone.  In  this  work  the  exact  magnitude  of 
the  adjustment  is  controlled  by  a  proposed  design  adjustment  vector  fi.  This  vector  is  a 
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set  of  scaling  factors,  much  like  a,  that  can  be  selected  to  make  small  adjustments  to  the 
nominal  parameter  values  po  of  a  candidate  solution.  One  element  of  the  design 
adjustment  vector  p  is  established  for  each  element  in  the  vector  po,  yielding  p  = 

{Pu...A,-A}  with-l<A<l. 

As  with  the  a  vector,  the  values  for  p  are  typically  taken  to  be  continuous.  If  only 
discrete  adjustments  are  possible  for  the  kth  parameter  of  a  system  the  corresponding  /4 
can  be  limited  to  discrete  options.  For  each  parameter  pk,  the  associated  pk  value  scales 
the  predetermined  maximum  allowable  adjustment  in  that  parameter  Spk.  dp  is  a  vector  of 
values  specified  by  the  designer  based  on  specific  system  limitations  and  should  be  of  the 
same  order  of  magnitude  as  the  original  uncertainty  levels  in  Ap.  For  example,  one 
parameter  of  a  notional  power  tool  design  may  be  the  thickness  of  wire  used  to  connect 
the  tooFs  electrical  supply  to  its  motor.  For  a  candidate  design  under  consideration  the 
designer  may  specify  10  AWG  wire  (American  Wire  Gage,  approximately  0.1  inches  in 
thickness).  However,  the  designer  may  know  that  the  choice  to  use  10  AWG  wire  was 
arbitrarily  made  based  on  other  factors  and  may  suspect  that  the  nominal  wire  thickness 
could  actually  be  adjusted  by  ±0.01  inches  without  affecting  the  electrical  system  of  the 
tool,  which  would  amount  to  using  9  AWG  (0.11  in)  or  11  AWG  (0.09  in)  wire.  As  such 
the  designer  would  then  select  dpwire  to  be  0.01  inches.  In  this  work  Spk  is  simply  taken  to 
be  ten  or  twenty  percent  of  its  nominal  value  po,k  (depending  on  the  engineering  system  of 
interest),  but  any  assumed  max  adjustment  is  acceptable  provided  the  max  range  of  the 
adjustment  is  sufficiently  small,  as  discussed  previously. 

When  considering  uncertain  parameters  whose  nominal  values  are  adjustable,  the 
effects  of  those  parameters  on  a  system’s  outputs  depends  on  the  nominal  parameter 
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values  po,  the  p  values  associated  with  each  parameter,  the  assumed  maximum 
adjustment  values  Sp  and  a  given  level  of  parameter  uncertainty  as  described  in  Eqn. 

(5.2) .  Again  the  vector  pv  is  used  to  denote  a  specific  realization  of  the  uncertainty  in  p , 
this  time  bounded  by  the  uncertain  interval  defined  by  Ap.  /?  and  Sp  as  shown  in  Eqn. 

(5.2) : 

/,GO  i=U/ 
gjipj 

for  all  pr  ^ 5 

where  {(Po  +  P  °Sp)~  Ap)<  p„  <  ((Po  +  P  c  fy)+  Ap) 

It  can  be  seen  in  Figure  5.3  that  small  changes  to  nominal  parameter  values  can 
ensure  the  feasibility  of  a  candidate  design  for  a  given  level  of  parameter  uncertainty. 

[(p0+  p  °  Sp)  -  Ap,  (p0  +  p  o Sp)  +  Ap) 


Figure  5.3:  Parameter  Adjustment  Mapping 

There  is  the  possibility  that  for  some  problems  no  combination  of  uncertainty 
reduction  or  small  adjustments  on  nominal  parameter  values  will  be  sufficient  to  ensure 
feasibility.  This  possibility  is  highly  problem  dependent  and  in  those  cases  a  designer 
can  enlarge  the  elements  of  the  maximum  allowable  adjustment  vector  Sp. 
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5.2.2  Objective  and  Constraint  Uncertainty  Mapping 


As  discussed  generally  in  Chapter  2,  in  order  to  effectively  manage  input 
uncertainty  in  an  effort  to  improve  output  function  performance  (i.e.,  to  reduce  the 
variation  in  the  outputs),  it  is  necessary  to  measure  and  quantify  objective  and  constraint 
function  variations  separately  as  a  function  of  a  and  /?.  In  order  to  do  this,  Li  et  al.’s 
work  [2009a]  is  extended  here  to  include  both  objective  functions/  as  well  as  constraint 
functions  gj  and  to  include  the  effects  of  parameter  adjustments  in  addition  to  parameter 
uncertainty  reduction.  Two  metrics,  Rf  and  Rg,  are  used  for  measuring  the  variations  in 
objective  and  constraint  functions  respectively  in  this  chapter.  The  formulation  of  these 
metrics  is  slightly  different  then  used  in  the  previous  chapter  due  to  the  inclusion  of  the  /? 
vector,  which  was  not  considered  in  MICOSA. 


Rf  —  max  ||A/|| 


where  A/,  = 


i  =  1,...,  / 


(G>.  +  P  °  Pp)~  a°&p)<pv  <  (( p0  +  P  °  5p)+  a  o  A p) 


R  =  max  (max  Ag) 


where  A  g  = 


j  =  1 .  J 


((p„  +  P  0  Sp)-a  o  A p)<  pv  <  {{p0  +  p  o  8p)+  a  °  A p) 


As  shown  in  Eqn.  (5.3)  objective  function  variations  are  normalized  by  the  nominal 
design’s  objective  function  values  and  then  the  largest  variation  is  found  using  a  II. lb 
norm;  however  a  II. IL  norm  is  also  acceptable  if  desired.  In  Eqn.  (5.4)  Rg  is  calculated  as 
the  maximum  value  of  all  constraint  function  values,  normalized  by  the  candidate 
design’s  nominal  constraint  function  values.  A  simple  maximization  is  used  in  Eqn.  (5.4) 
to  find  the  largest  constraint  deviation  from  the  nominal  and  ensures  that  any  positive  Rg 
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value  will  indicate  an  infeasible  input  uncertainty  interval  [(/?o  +  ft  °Sp)  -  a  °  Ap  ,  (po  +  ft  ° 
Sp)  +  a  o  A/?].  Since  the  Rf  and  Rg  metrics  measure  output  variations  given  input 
uncertainty  levels,  both  a  and  ft  values  must  be  specified  for  the  system  of  interest  prior 
to  evaluating  Eqns.  (5.3)  and  (5.4).  Once  a  and  ft  values  are  given,  Rf  and  Rg  values  in 
Eqns.  (5.3)  and  (5.4)  can  then  be  calculated  by  performing  a  single  objective  optimization 
to  find  the  worst  case  variation  in  the  system’s  objectives  (Rf)  and/or  constraints  (Rg). 
The  decision  variables  in  each  of  these  optimizations  are  the  values  for  the  system’s  input 
parameters  (denoted  by  pv )  bounded  by  a  predetermined  level  of  uncertainty  in  each  of 
those  parameters  (specified  by  a  and  ft  values).  In  this  work,  Rf  and  Rg  are  mathematical 
metrics  that  quantify  the  resulting  variations  in  multiple  system  outputs  as  a  single  scalar 
value  for  either  the  system’s  objectives  (Rf)  or  constraints  (Rg).  Figure  5.4  depicts  this 
relationship  between  changes  in  a  and  Rf  and  Rg  for  demonstration  purposes.  A  similar 
figure  can  be  drawn  for  /fyand  Rg  as  a  function  of  ft  only,  or  both  a  and  ft  simultaneously. 


For  many  engineering  systems  it  is  possible  that  uncertainty  reduction  and/or 
design  adjustments  may  not  be  possible  for  one  or  more  of  nominal  parameters.  Such 
limitations  can  be  addressed  by  adjusting  the  upper  and/or  lower  bounds  of  the 
corresponding  elements  of  a  and  ft. 
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5.2.3  Parameter  Adjustment  Cost  Metric 


In  most  engineering  systems  uncertainty  reduction  and/or  small  design  adjustments 
are  not  always  possible  without  additional  penalty,  cost,  or  investment.  For  example,  a 
better  machining  process  could  produce  a  more  accurate  (less  uncertain)  dimension  for  a 
part,  but  only  if  a  designer  or  manager  were  willing  to  make  the  cost  investment  required 
to  purchase  better  equipment.  Similarly,  changing  the  nominal  design  parameters  for  a 
candidate  design  may  not  be  completely  desirable  as  small  changes  may  require  other 
related  parts  or  complementary  sub-systems  to  be  redesigned  as  well,  incurring  more  time 
and  costs.  In  order  to  deal  with  these  realities  the  following  generic  formulation  for 
quantifying  these  costs  is  provided  in  Eqn.  (5.5).  This  metric  I  (short  for  investment)  is 
an  extension  of  the  Investment  metric  developed  by  Li  et  al.  [2009a]  and  used  in  the  last 
two  research  thrusts,  modified  here  to  measure  both  the  uncertainty  reduction 
requirements  and  design  adjustments. 


X(w,(l-«J+ w2|^|) 

/ =-i -  k  =  l,...,K 

k 

where  0  <  (w1 ,  w2 )  <  1  and  vv,  +  w,  =  I 


(5.5) 


I  can  be  used  as  a  measure  of  the  cost  associated  with  those  changes  made  to  a 
nominal  solution  in  the  absences  of  any  actual  cost  information  for  a  given  system.  The 
two  weighting  factors  in  Eqn.  (5.5),  w\  and  W2,  can  be  used  to  adjust  the  relative  costs 
associated  with  uncertainty  reduction  and  nominal  design  adjustment  in  accordance  with 
the  designer’s  preferences.  For  this  work  w i  =  0.75  and  W2  =  0.25  are  chosen, 
corresponding  to  an  arbitrarily  determined  preference  for  design  adjustment  over 
uncertainty  reduction  (accomplished  by  placing  a  greater  weight,  or  penalty,  on 
uncertainty  reduction). 
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5.3  DISA  APPROACH 


In  this  section  the  Design  Improvement  by  Sensitivity  Analysis  (DISA)  approach  is 
formulated.  DISA  seeks  to  efficiently  determine  the  optimal  combination  of  uncertainty 
reductions  and/or  small  design  adjustments  required  in  order  to  produce  the  best  possible 
reduction  in  objective  function  variation  while  ensuring  the  feasibility  of  a  design  under 
uncertainty.  This  is  accomplished  by  solving  two  bi-objective  optimization  problems  in  a 
sequential  fashion.  The  first  optimization  determines  a  Pareto  set  of  uncertainty 
reduction  (a)  options  in  an  effort  to  reduce  objective  function  variation  first.  In  the 
context  of  this  approach  uncertainty  reduction  is  preferable  since  it  does  not  require 
changes  to  the  nominal  parameter  values  of  the  predetermined  candidate  design.  Then,  if 
necessary,  a  second  optimization  problem  is  solved  repeatedly,  once  for  each  Pareto 
solution  of  interest  produced  during  the  first  stage  optimization,  in  an  effort  to  determine 
the  optimal  design  adjustment  (/})  required  to  ensure  feasibility.  If  desired,  this  second 
stage  optimization  can  be  performed  using  a  set  of  meta-models  (one  for  each  system 
output)  created  using  system  data  retained  during  the  first  stage  optimization  problem. 

The  decision  to  formulate  the  DISA  approach  using  a  sequential  algorithm  (as 
opposed  to  the  seemingly  more  intuitive  all-at-once  approach  where  both  a  and  fi  would 
be  optimized  simultaneously)  was  made  for  a  specific  set  of  reasons.  Indeed  DISA 
intentionally  does  not  attempt  to  obtain  solutions  that  are  optimal  with  regard  to  both  a 
and  p  simultaneously,  accepting  the  possibility  of  producing  inferior  solutions  in  order  to 
provide  an  approach  that  has  the  following  three  important  advantages  over  the  all-at- 
once  approach.  First,  it  is  possible  that  uncertainty  reduction  procedure  alone  could  be 
enough  to  ensure  feasibility.  These  types  of  solutions  could  be  very  attractive  to  some 
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designers  but  may  potentially  be  suppressed  (and  therefore  missed)  in  an  all-at-once 
approach.  Secondly,  a  two-stage  approach  allows  a  designer  to  only  perform  the  second 
stage  optimization  on  a  subset  of  uncertainty  reduction  solutions  obtained  in  the  first- 
stage  optimization  that  are  of  interest,  providing  the  designer  with  further  control  over  the 
approach  and  the  type  of  results  that  it  produces.  The  third  and  most  compelling  reason 
for  formulating  DISA  in  this  way  is  to  allow  for  greater  computational  efficiency. 
Solving  DISA  using  two  sequential  stages  makes  it  possible  to  use  analysis  data  obtained 
in  the  first  stage  optimization  to  perform  the  second  stage  optimizations  (recall  that  the 
second  stage  optimization  may  be  solved  repeatedly)  using  surrogate  approximations  as 
opposed  to  additional  function  calls.  In  this  chapter  a  function  call  is  defined  as  the 
evaluation  of  one  specific  input  parameter  vector  pv  for  its  output  function  (i.c.,/  and  g) 
values  obtained  from  a  computational  analysis  or  simulation  model.  See  Chapter  2  for 
details  on  surrogate  approximation. 

To  take  advantage  of  the  efficiency  gains  mentioned  above  all  uncertain  inputs  to 
the  system  must  be  sufficiently  sampled  over  their  ranges  of  uncertainty  during  the  first 
stage  of  the  approach.  This  condition  will  be  met  in  those  cases  when,  after  the  first  stage 
of  the  approach,  all  possible  values  for  any  uncertain  parameters  to  be  considered  in  the 
second  stage  are  guaranteed  to  fall  within  the  original  maximum  uncertain  intervals  for 
those  parameters.  This  is  because  the  first  stage  of  the  approach  varies  all  uncertain 
input  parameters  over  their  maximum  uncertain  intervals  in  search  of  uncertainty 
reduction  opportunities  as  shown  in  Eqns.  (5.3)  and  (5.4),  generating  a  wealth  of  function 
call  (system  outputs  as  a  function  of  pv )  information.  Clearly,  for  many  systems  the 
parameter  space  may  not  be  sufficiently  sampled  during  the  first  stage  of  the  algorithm  as 
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discussed  above  and  additional  data  will  be  required  in  order  to  build  effective  meta¬ 
models  of  the  system’s  objective  and  constraint  functions  prior  to  searching  for  optimal 
adjustments  to  the  nominal  values  of  uncertain  parameters.  This  does  not  preclude  the 
use  of  the  DISA  approach  and  simply  requires  that  any  additional  required  function  call 
information  be  obtained  after  the  first  stage  problem  but  prior  to  building  any  meta¬ 
models  for  use  in  the  second  stage.  This  case  will  be  addressed  in  detail  in  Section  5.3.2 
and  in  Section  5.4  through  the  engineering  examples. 

Additionally,  it  should  be  pointed  out  that  using  meta-models  within  the  second 
stage  of  the  DISA  approach  is  not  without  its  own  set  of  potential  drawbacks  which  are 
well  known  and  beyond  the  scope  of  this  work.  Prior  to  employing  the  meta-model 
assisted  DISA  approach  the  user  should  become  familiar  with  the  strengths  and 
weaknesses  associated  with  surrogate  approximation  by  reviewing  the  literature  for 
details  (e.g.,  [Martin  and  Simpson,  2005]). 

5.3.1  DISA  Formulation 

The  DISA  approach  begins  with  a  candidate  design  solution,  defined  by  the 
parameter  vector  p0  for  a  system  comprised  of  multiple  objective  functions  /  and 
constraint  functions  g.  The  designer  then  determines  which  parameters  have  uncertainty 
and/or  whose  nominal  values  can  be  adjusted,  including  their  associated  uncertainty 
intervals  and  maximum  design  adjustments,  A p  and  dp,  respectively.  A  MOSA  based 
unconstrained  optimization  problem  (i.e.,  Eqn.  (2.3)  with  I  substituted  for  Investment)  is 
then  performed  to  minimize  both  the  variation  in  objectives  (Rf)  and  the  metric  /  as 
defined  in  Eqn.  (5.5).  Recall  that  the  decision  variables  in  this  first  stage  optimization 
problem  are  only  the  elements  of  the  uncertainty  reduction  vector  a,  and  since  Eqn.  2.3 
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does  not  contain  /?  (since  MOSA  does  not  consider  design  adjustments),  /?  is  taken  to  be  a 

zero  vector  in  the  first  stage.  The  optimization  can  be  conducted  using  a  population 

based  optimizer,  such  as  a  Multi-Objective  Genetic  Algorithm  (MOGA)  [Deb,  2001]  and 

during  each  generation  all  analysis  data  from  the  simulation  model(s)  are  retained.  Recall 

that  this  retained  data  is  in  the  form  of  f(pv)  and  gj(pv )  data  can  be  obtained  when  Eqn. 

(5.3)  is  solved  within  Eqn.  (2.3),  assuming  that  the  gj(pv)  constraint  functions  are  also 

evaluated.  Following  the  completion  of  the  first  stage  optimization  a  Pareto  set  of  Np 

optimal  uncertainty  reduction  vector  a*  solutions  will  be  obtained.  Since  this  first  stage 

optimization  problem  is  performed  in  an  unconstrained  fashion  (recall  that  MOSA  is  an 

unconstrained  problem)  the  Pareto  solutions  produced  cannot  ensure  the  feasibility  of  the 

candidate  design.  Since  uncertainty  reduction  alone  may  not  be  enough  to  ensure 

feasibility  the  second  stage  optimization  is  performed  by  solving  Eqn.  (5.6),  once  for 

each  uncertainty  reduction  solution  of  interest  obtained  in  the  first  stage  problem. 

min  RAp  I  a*) 

P 

min  I{p  I  aj) 

'  (5.6) 

s.t. 

Rg(p\a*)<0 

The  second  stage  is  solved  repeatedly,  once  for  each  Pareto  a,*  solution  of  interest, 
i  =  (N]  <Np).  A  designer  may  wish  to  perform  the  second  stage  optimization 

procedure  for  all  «*  solutions  obtained  in  the  first  stage  if  desired.  It  is  also  permissible  to 
limit  the  number  of  a*  solutions  N\  considered  in  the  second  stage  optimization  problem 
if  a  small  subset  of  first  stage  solutions  is  more  attractive.  Selecting  multiple  solutions  of 
interest  will  increase  the  range  of  possible  feasible  solutions  produced  by  the  approach, 
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providing  the  designer  with  a  larger  set  of  design  adjustment  options.  In  each  of  the 
second  stage  optimizations,  i  =  the  a,*  value  of  interest  is  held  constant.  As 

previously  discussed,  as  long  as  Rg  is  less  than  or  equal  to  0  the  corresponding 
combination  of  p  and  a*  values  will  produce  a  feasible  design.  Simultaneously 
minimizing  Rf  and  I  again  creates  a  natural  tension,  this  time  between  objective  function 
variation  reduction  due  to  design  adjustments  and  the  required  magnitude  of  those  design 
adjustments.  It  is  important  to  note  that  the  second  stage  solves  for  what  is  defined  as 
/?*l a,*  (i.e.,  /?*  given  a,*),  not  the  more  general  independent  /?*,  which  is  an  important 
distinction.  It  is  possible  that  for  the  ith  first  stage  solution  considered  in  the  second  stage 
no  combination  of  fi*\ a,*  can  produce  a  feasible  solution  (Rg  <  0)  and  as  a  result  no 
Pareto  solutions  will  be  produced.  In  this  case  it  is  up  to  the  designer  to  select  an 
alternate  «,*  solution  from  the  first  stage  to  investigate  and  further  drives  the  need  for  the 
designer  to  select  multiple  a;*  solutions  of  interest. 

If  the  DISA  approach  is  solved  in  the  manner  described  above  using  actual  analysis 
functions  to  repeatedly  solve  Eqn.  (5.6),  the  computational  effort  (in  terms  of  the  number 
of  function  calls)  can  become  a  problem  if  numerous  p  optimization  problems  have  to  be 
solved.  In  order  to  improve  the  computational  efficiency  of  the  DISA  approach,  Eqn. 
(5.6)  is  restated  by  using  meta-models  to  obtain  the  values  of  Rf,  Rg  and  /  within  the 
second  stage  optimization.  The  data  for  creating  these  meta-models  comes  from  the  data 
retained  during  the  first  stage  uncertainty  reduction  optimization  problem  as  previously 
discussed.  Before  beginning  the  optimization(s)  to  determine  corresponding  p  values, 
meta-models  for  all  functions  f  and  gj  are  created  and  then  used  in  lieu  of  the  actual 
simulation  functions  to  determine  approximated  Rf,  Rg  and  /  values.  Prior  to  building 
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those  meta-models  it  is  important  to  check  the  maximum  and  minimum  pv  values  retained 
during  the  first  stage  and  compare  them  to  the  maximum  and  minimum  pv  values  that 
could  be  evaluated  in  the  second  stage  (which  can  be  determined  based  on  a,*  and  dp).  If 
there  is  any  possibility  that  any  of  the  second  stage  pv  value  could  fall  outside  the  range 
of  retained  data,  then  additional  function  calls  are  needed  and  should  be  obtained  and 
included  in  the  meta-models.  Eqn.  (5.7)  formulates  this  meta-model  assisted  second 
stage  optimization  to  solve  for  /?*l a,*  Pareto  solutions.  Recall  that  all  f(pv)  and  gj(pv) 
information  generated  during  the  first  state  procedure  is  retained,  despite  the  fact  that  the 
first  stage  optimization  does  not  check  for  constraint  violation,  making  it  possible  to 
generate  meta-models  of  the  systems’  constraint  functions. 

min  R A B\  a  ) 

p  J  '  1 

min  i{j3\a) 

B 

,  (5-7) 

s.t. 

Rg{/3\oci)<0 

Figure  5.5  depicts  both  formulations  of  the  DISA  approach  for  comparison 
puiposes.  Figure  5.5(a)  depicts  the  general  formulation  which  does  not  require  meta¬ 
models  of  the  system  output  functions  and  solves  Eqn.  (2.3)  and  (5.6)  sequentially. 
Figure  5.5(b)  shows  the  more  efficient  meta-model  assisted  DISA  approach  which 
involves  solving  Eqn.  (2.3)  and  Eqn.  (5.7). 
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Figure  5.5:  DISA  Approach  (a)  Without  Meta-Models  and  (b)  With  Meta-Models 


5.3.2  DISA  results 

The  DISA  approach  will  produce  solutions  with  the  following  properties,  as  shown 
in  Figure  5.6.  The  first  stage  Pareto  a*  values  will  optimally  reduce  the  uncertainty 
levels  in  the  system.  However,  this  first  stage  procedure  will  not  necessarily  ensure  that 
the  resulting  design  system  will  be  feasible  (note  that  in  Figure  5.6  if  only  uncertainty 
reduction  is  considered  the  constraint  gi  is  still  violated).  As  a  result  the  subsequent 
optimal  selection  of  corresponding  p*  values  for  each  a,*  solution  are  needed  to  ensure 
feasibility  of  the  design  under  uncertainty  with  a  minimal  change  in  objective 
performance. 
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Figure  5.6:  Sample  DISA  Results  in  Two  Dimensions 

Figure  5.7  depicts  the  cascading  flow  of  information  produced  by  this  approach. 
As  can  be  seen  on  the  lower  left  side  of  the  figure  the  first  stage  optimization  finds  a 
Pareto  set  of  a*  solutions  which  optimally  reduce  uncertainty  levels.  Then,  for  each  a* 
solution  of  interest  produced  (one  of  which  is  depicted  on  the  diagram)  a  second 
optimization  is  performed  using  meta-modeling  (or  actual  analysis  functions  if  desired)  to 
produce  an  additional  Pareto  set  of  options,  this  time  adjusting  the  nominal  values  of 
uncertain  parameter  in  an  effort  to  ensure  feasibility  given  the  predetermined  optimal 
uncertainty  reduction  levels. 
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Figure  5.7:  Sample  Propagation  of  DISA  Result 

If  analysis  functions  (or  simulation  models)  are  used  for  both  stages  of  the  DISA 
algorithm,  as  shown  in  Figure  5.5(a),  the  computational  effort  would  be  equal  to  the 
number  of  function  calls  required  to  solve  the  a  problem,  M,  times  the  number  of 
function  calls  required  to  solve  each  p  problem,  N,  times  the  number  of  times  the  p 
problem  needs  to  be  solved,  N\  in  the  worst  case,  resulting  in  a  total  computational  effort 
of  O(MNNi)  calls.  If  meta-models  are  used  to  approximate  the  analysis  function  for  all  p 
problem  optimizations  performed,  as  shown  in  Figure  5.5(b),  the  computational  savings 
would  amount  to  the  sum  total  of  all  function  calls  needed  for  all  p  problems  required  to 
be  solved,  or  0{NN\)  calls.  Clearly  this  is  a  significant  reduction  in  computational  effort. 
However,  when  using  the  meta-model  assisted  approach  it  is  important  to  verify  that  the 
parameter  space  is  sufficiently  sampled  during  the  first  stage  optimization  in  order  to 
produce  valid  meta-models  over  the  entire  range  of  potential  design  adjustments,  as 
pointed  out  at  the  beginning  of  this  section.  If  the  range  of  permissible  design 
adjustments  for  a  specific  problem  is  sufficiently  smaller  than  the  uncertainty  levels 
presented  in  the  system  no  additional  information  is  required  as  the  parameter  space  will 
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be  sufficiently  sampled  during  the  first  stage  optimization.  If  the  range  of  permissible 
design  adjustments  is  larger  than  the  area  of  the  parameter  space  sampled  during  the  first 
stage  optimization  then  additional  sampling  data  must  be  collected  after  the  first  stage 
optimization  in  order  to  construct  valid  meta-models,  as  previously  discussed.  In  this 
case  the  computational  effort  would  clearly  increase  by  the  number  of  function  calls  P 
required  to  adequately  sample  the  neglected  portion  of  the  parameter  space.  In  this  case 
the  resulting  computational  effort  would  be  0(M+P),  which  is  still  significantly  less  than 
the  O(MNNi)  function  calls  required  if  no  meta- modeling  is  employed.  This 
differentiation  will  be  demonstrated  in  the  next  section  via  the  first  example.  The  exact 
number  of  function  calls  required  in  this  case  (P),  however,  should  be  determined  by  the 
designer  based  on  the  size  of  the  parameter  space  that  needs  to  be  sampled. 

5.4  EXAMPLES  AND  RESULTS 

In  order  to  demonstrate  the  capabilities  and  efficiency  of  the  DISA  approach,  two 
engineering  examples  of  varying  complexity  are  presented.  In  both  examples  a  single 
candidate  design  was  obtained  beforehand  using  a  deterministic  (no  uncertainty 
considered)  optimization  procedure.  The  nominal  parameter  values  for  the  candidate 
design  were  located  on  or  near  the  boundaries  of  the  corresponding  system’s  feasible 
domain.  As  a  result,  the  nominal  designs  will  become  infeasible  if  the  input  parameters 
are  acknowledged  to  be  uncertain.  All  parameter  uncertainty  intervals  in  these  examples 
are  considered  to  be  reducible  and  all  nominal  parameter  values  considered  are  assumed 
to  be  adjustable  by  a  small  amount.  In  other  words  there  will  be  associated  selectable  a* 
and  Pk  values  for  each  parameter  considered.  For  the  examples  presented  in  this  section 
the  problems  are  solved  twice  in  order  to  validate  the  capability  of  the  meta-model 
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assisted  DISA  approach,  first  obtaining  solutions  using  the  DISA  approach  of  Figure 
5.5(a)  and  then  again  using  the  meta-model  assisted  DISA  approach  detailed  in  Figure 
5.5(b). 

5.4.1  Thin-Walled  Tube  Design 

The  first  example  presented  is  a  simple  two  dimensional  problem  adapted  from 
Arora  [2004]  to  consider  two  objectives,  chosen  so  that  the  solutions  produced  can  be 
presented  graphically.  Figure  5.8  depicts  the  simple  hollow  tube  system  under 
consideration  which  is  described  by  its  length,  L,  radius,  R.  wall  thickness,  t,  and  the 
compression  load  it  supports,  P. 


F 


Figure  5.8:  Tube  Design  Model 

Determining  the  optimal  dimensions  for  the  parameters  R  and  t  without  uncertainty 
can  be  accomplished  by  solving  the  following  bi-objective  optimization  problem: 


min  /,  =  IphnRt 


min  f,=  — 

R.,  Jl  f 

s.t. 


(5.8) 


R  2:  0,  r  2  0 
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The  optimization  problem  in  Eqn.  (5.8)  seeks  to  minimize  the  weight  (/)  and  the 
radius  to  thickness  ratio  (fz)  of  the  tubular  column  subject  to  stress  (gi),  buckling  (gz)  and 
non-negativity  constraints.  Eqn.  (5.8)  contains  the  following  constant  parameters:  p  = 
7833  kg/m3  (material  density),  h  =  5  m  (tube  length),  F  =  10,000  N  (tube  loading),  amax  = 
248  MPa  (max  stress)  and  E  =  207  GPa  (Young’s  modulus).  One  deterministic  optimal 
solution  to  this  problem  is  R  =  0.1558  (m)  and  t  =  0.0412  (m),  which  for  this  problem  can 
be  obtained  graphically.  For  this  solution  the  nonlinear  constraints  gi  and  gz_  are  both 
active.  Based  on  this  fact  it  is  clear  that  if  the  input  parameters  R  and  t  possess  any 
uncertainty  the  only  way  to  ensure  the  feasibility  of  the  design  would  be  to  completely 
eliminate  all  uncertainty,  which  is  usually  not  an  option.  As  a  result  the  DISA  approach 
can  be  used  for  this  problem  to  determine  an  optimal  combination  of  uncertainty 
reduction  and  small  design  adjustments  to  ensure  both  feasibility  and  minimal 
performance  variations  with  respect  to  the  two  original  objective  functions,/!  and/2  in 
Eqn.  (5.8). 

The  uncertainty  ranges  A p  for  each  parameter  (i.e.,  R  and  t)  were  set  to  be  ±20%  of 
each  parameter’s  nominal  value,  as  given  above.  Additionally  each  parameter  was 
allowed  to  be  adjustable  (dp)  by  ±10%  from  its  nominal  value.  These  values  were 
determined  intuitively  for  the  purpose  of  demonstrating  the  capabilities  of  the  approach, 
but  for  specific  problems  designers  should  set  A p  and  dp  using  system  information. 
Based  on  these  selected  ranges  for  A p  and  dp  the  allowable  design  adjustments  will  be 
guaranteed  to  be  within  the  limits  of  the  parameter  uncertainty  considered  for  each 
parameter  provided  at  least  50%  uncertainty  reduction  (a  <  0.5)  in  the  first  stage  problem. 
This  is  because  a  50%  reduction  in  uncertainty  will  result  in  a  maximum  retained 
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uncertainty  of  ±10%  of  the  nominal  parameter  values,  which  will  be  guaranteed  to  fall 
within  the  original  uncertain  region  (±20%  of  nominal)  even  if  p  adjusts  po  by  the 
maximum  10%  adjustment.  The  DISA  approach  was  implemented  using  a  MOGA 
optimizer  and  produced  the  results  shown  in  Figures  5.9  and  5.10. 


Figure  5.9:  p  Results  for  Three  Sample  a,*  Solutions 
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Figure  5.10:  Adjusted  Uncertain  Solution  in  the  Parameter  Space 
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Figure  5.9  is  an  objective  space  (R/V s.  I)  plot  of  the  a*  Pareto  solutions,  along  with 
three  sets  of  p  Pareto  solutions  corresponding  to  three  selected  a*  solutions  (which  have 
been  highlighted  in  grey  filled  circles  in  Figure  5.9).  Figure  5.10  is  a  plot  of  the  same  p 
Pareto  solutions  shown  in  Figure  5.9,  but  plotted  in  the  parameter  space  of  the  tube 
design  problem  in  order  to  show  their  relationship  to  the  problem’s  objective  and 
constraint  function  contours.  In  Figure  5.10  the  nominal  design’s  original  input 
parameter  uncertainty  region  and  the  retained  uncertainty  regions  for  two  representative  p 
solutions  (called  out  with  a  grey  fill)  are  plotted  as  rectangles  around  those  solutions. 
This  has  been  done  to  clearly  show  that  the  original  candidate  design  is  infeasible  due  to 
input  parameter  uncertainty  and  that  the  feasibility  of  the  resulting  solutions  produced  by 
the  DISA  approach  is  ensured.  In  both  figures  the  solutions  obtained  with  actual  analysis 
function  calls  are  shown  in  black  outlines  while  those  obtained  using  kriging  meta¬ 
models  of  the  analysis  functions  are  shown  in  grey  outlines.  The  geometric  shapes  of  the 
solution  point  markers  (triangles,  squares,  and  diamonds)  are  the  same  for  corresponding 
solutions  in  both  Figures  5.9  and  5.10  for  comparison  purposes.  For  instance,  the  square 
shaped  solutions  in  Figure  5.10  are  the  parameter  space  plots  of  the  square  shaped  Pareto 
points  in  Figure  5.9.  Recall  that  all  a  solutions  were  obtained  using  actual  analysis 
functions.  In  Figure  5.10  the  original  objective  contours  are  depicted  in  black  dashed 
lines  (to  show  that  the  DISA  approach  does  in  fact  minimize  the  variation  in  the  original 
system  objectives),  the  constraint  function  contours  are  depicted  in  bold  and  dashed  black 
lines  and  the  nominal  design  solution  is  depicted  by  a  white  filled  circle.  It  should  be 
noted  that  when  both  black  and  grey  solution  cannot  be  clearly  distinguished  on  one  of 
the  corresponding  figures  it  is  because  there  is  excellent  agreement  between  the  two 
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solution  methods  (actual  analysis  functions  in  black  outlines  and  kriging  meta-model 
assisted  solutions  in  grey  outlines). 

As  can  be  seen  in  Figure  5.9,  each  p  Pareto  solution  set  generated  has  larger  Rf  and 
I  values  than  the  a*  solution  used  to  generate  those  corresponding  p  solutions.  This  is 
because  the  parameter  adjustments  made  to  the  candidate  design  in  the  second  stage 
problem  clearly  increase  both  Rf  and  I  when  p  values  are  nonzero,  as  seen  in  Eqn.  (5.3) 
and  Eqn.  (5.5)  respectively.  This  is  the  price  of  ensuring  feasibility.  The  candidate 
design  is  clearly  on  two  active  constraint  functions  as  shown  in  Figure  5.10.  As  a  result 
all  DISA  solutions  to  this  problem  have  been  moved  away  from  the  constraint  boundaries 
by  some  small  amount,  resulting  in  nonzero  p  values.  The  exact  amount  of  parameter 
adjustments  required  is  a  function  of  the  retained  uncertainty  in  the  system  following  the 
first  stage  problem.  As  the  amount  of  retained  uncertainty  in  the  system  increases,  as 
shown  by  a  decrease  in  I  in  the  a  Pareto  on  Figure  5.9  (i.e.,  from  a i*  to  a2*),  larger 
adjustments  to  the  nominal  parameter  values  are  required.  This  can  be  seen  in  Figure  5.10 
as  the  P  solutions  corresponding  to  a2*  are  further  away  from  the  nominal  solution  than 
those  corresponding  to  «i  *,  and  the  black  outlined  triangle  p  solution  (corresponding  to 
a  i  *)  has  a  smaller  retained  uncertainty  range  than  the  grey  outlined  square  point 
(corresponding  to  a?*). 

As  shown  in  Figures  5.9  and  5.10,  for  each  of  the  solutions  presented  (i.e.,  from  ai* 
to  «3*),  the  meta-model  assisted  p  solutions  do  not  match  the  analysis  function  results 
perfectly,  but  they  do  show  general  agreement,  suggesting  that  any  effective  meta¬ 
modeling  technique  could  produce  solutions  without  the  need  for  additional  function  calls 
after  first  solving  the  a  problem  in  this  example.  In  this  example  the  a  problem  required 


109 


61,707  function  calls  while  each  of  the  three  /?  problems  solved  using  the  DISA 
approach  of  Figure  5.5(a)  required  178,120,  180,255  and  180,255  function  calls 
respectively.  Recalling  that  the  meta-model  assisted  DISA  solution,  obtained  using  the 
approach  formulation  of  Figure  5.5(b),  did  not  require  any  additional  function  calls  than 
what  was  previously  obtained  to  solve  the  a  problem  and  thus  it  can  be  concluded  that  the 
meta-models  assisted  formulation  leads  to  a  total  computational  savings  of  538,630 
function  calls. 

If  the  above  problem  is  restated  using  ±15%  and  ±20%  of  nominal  as  the  ranges  of 
parameter  uncertainty  A p  and  maximum  design  adjustments  dp  respectively,  clearly  the 
range  of  potential  design  adjustments  will  fall  outside  of  the  region  of  the  parameter 
space  sampled  during  the  first  stage  optimization.  As  a  result,  between  the  first  and 
second  stage  optimizations  the  parameter  space  needs  to  be  sampled  a  bit  more  to  ensure 
adequate  coverage  in  order  to  build  valid  meta-models  for  design  adjustments.  The  DISA 
approach  was  applied  to  the  problem  again,  this  time  using  these  new  ranges.  The 
computational  effort  for  the  a  and  fS  problems  was  comparable  to  the  original  example, 
but  this  time  an  additional  1,017  function  calls  were  used  to  sample  the  region  of  the 
parameter  space  not  considered  during  the  first  stage  optimization.  For  this  case  the 
results  shown  in  Figures  5.11  and  5.12  are  produced  (which  use  the  same  notation  as 
Figures  5.9  and  5.10  above). 
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Figure  5.11:  Alternate  Problem  Pareto  Frontier 
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Figure  5.12:  Alternate  Problem  Adjusted  Uncertain  Solutions 

Notice  that  in  Figure  5.12  the  original  nominal  uncertain  region  is  much  smaller 
and  that  the  retained  uncertain  regions  for  the  solution  points  lie  outside  the  original 
uncertain  region.  However,  the  produced  results  from  kriging  assisted  DISA  (Figure 
5.5(b))  essentially  agree  with  the  results  produced  by  the  DISA  approach  using  actual 


analysis  functions  (Figure  5.5(a)).  Clearly,  a  limited  number  of  additional  function  calls 
can  produce  effective  results  in  the  cases  where  acceptable  design  adjustments  fall 
outside  of  the  region  of  the  parameter  space  considered  during  the  uncertainty  reduction 
optimization  problem. 

5.4.2  Angle  Grinder  Design 

To  demonstrate  the  properties  of  the  DISA  approach  with  a  more  complex  system, 
an  angle  grinder  model  used  by  Li  et  al.  [2009a]  (first  presented  by  Williams  et  al.  [2008] 
and  used  in  the  Chapter  3)  was  analyzed.  As  shown  in  Figure  5.13,  the  angle  grinder 
system  consists  of  a  battery  model,  a  motor  model  and  a  bevel  gear  model  containing 
approximately  30  input  parameters  and  numerous  outputs.  A  deterministic  optimal 
solution  that  produces  a  minimal  grinder  weight  and  a  maximum  available  power  was 
found  using  MOGA  prior  to  employing  the  DISA  approach  to  this  system.  Six  of  the 
model  parameters  were  considered  to  have  uncertainty  in  this  example.  For  specific 
details  on  the  grinder  model  see  Williams  et  al.  [2008], 

Stator  Radius  ( Ra ) 

Stator  Thickness  (ts) 

Gap  Length  (LGap) 


Stack  Length  (LStack) 

Copper  Wire  Area  (Aw) 

Copper  Wire  Resistance  (Rw) 

Figure  5.13:  Angle  Grinder  System 

Table  5.1  provides  the  settings  used  in  the  application  of  the  DISA  approach,  while 
the  nominal  values  for  the  6  parameters  considered  in  this  application  are  shown  below  in 
Table  5.2  (all  other  inputs  to  grinder  model  were  taken  as  deterministic  constants). 
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Table  5.1:  Grinder  Problem  Specifics 


Objectives 

2 

Constraints 

9 

Parameters 

6 

Uncertainty 

20%  of  nominal 

Max  Adjustment 

10%  of  nominal 

All  of  these  parameters  are  critical  elements  of  the  design  of  the  angle  grinder’s 
motor  sub-system.  These  6  parameters  were  analyzed  by  the  DISA  approach  and  were 
assumed  to  be  both  uncertain  and  adjustable  in  accordance  with  the  limits  given  in  Table 
5.1.  The  nominal  design  was  selected  such  that  four  constraints  g2,  gi  and  g4  were  very 
close  to  being  active.  These  constraints  all  represent  limitations  on  the  flux  density  of  the 
stator,  armature  and  air  gap  of  the  grinder’s  electric  motor. 

Table  5.2:  Grinder  Problem  Nominal  Parameters 


Parameters  Nominal  Values 


Stator  Radius  (R0) 

0.0012  m 

Stator  Thickness  ( ts ) 

0.0501  m 

Gap  Length  (LGap) 

0.0006  m 

Stack  Length  ( LStack ) 

0.0188  m 

Copper  Wire  Area  ( Aw ) 

0.036  mm2 

Copper  Wire  Resistance  ( Rw ) 

0.504  a 

Using  the  data  in  Tables  5.1  and  5.2  and  the  grinder  analysis  functions,  a  set  of 
DISA  results  were  obtained  using  a  MOGA  optimizer.  Figure  5.14  displays  three 
representative  sets  of  /?  results  for  the  grinder  problem  for  three  selected  a  Pareto 
solutions.  As  with  the  previously  presented  two-dimensional  example  fS  solutions  were 
determined  in  two  ways:  using  actual  analysis  function  model  and  using  kriging  meta¬ 
models.  Figure  5.14  contains  Rf  vs.  I  plots  for  both  the  results  of  the  first  stage 
optimization  and  three  sets  of  Pareto  solutions.  In  Figure  5.14,  all  a  Pareto 

solutions  are  presented  in  black  circles.  The  a,-*  solutions  considered  for  each  depicted 
/?*!«,*  Pareto  are  called  out  by  grey  filled  circles.  The  corresponding  /?  Pareto  solutions 
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obtained  using  the  analysis  function  are  presented  in  black  outlines  and  the  kriging  meta¬ 
model  assisted  results  are  presented  in  grey  outlines.  As  with  the  plots  in  the  previous 
example,  when  a  distinction  between  the  black  and  grey  solutions  cannot  be  distinguished 
on  the  figure  it  is  because  the  solutions  obtained  agree  directly.  Not  all  of  the  a  Pareto 
solutions  for  this  example  were  infeasible  following  the  first  stage  problem,  indicating 
that  for  some  cases  (7  out  of  15  a  Pareto  solutions)  the  feasibility  of  the  system  could  be 
achieved  by  uncertainty  reduction  alone  and  the  second  stage  procedure  was  unnecessary. 
However,  those  solutions  that  were  feasible  after  the  first  stage  procedure  also  required  a 
high  level  of  investment  (low  a  values)  which  could  potentially  be  undesirable  to  the 
designer.  For  those  solutions  that  were  infeasible  (as  indicated  by  positive  values)  the 
second  stage  procedure  of  the  DISA  approach  was  necessary.  Three  a,*  solutions  shown 
in  Figure  5.14  were  infeasible  yet  after  the  first  stage  procedure  and  thus  need  further 
adjustment  to  guarantee  feasibility. 


Figure  5.14:  Grinder  (i  Results  for  Various  a\*  Solutions 
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As  can  be  seen  in  Figure  5.14,  the  p  solutions  corresponding  to  two  selected  a\* 
and  ai*  overlap  quite  a  bit.  For  this  reason  Figure  5.15  shows  just  the  first  stage  solution 
a\ *  and  its  corresponding  /?  Pareto.  Several  observations  can  be  made  from  Figures  5.14 
and  5. 15.  First,  in  order  to  ensure  feasibility  additional  cost  is  necessary  as  all  ft  solutions 
have  higher  /  values  than  «t*.  Secondly,  the  trade-off  between  /  and  Rf  is  very  clear  for 
the  fi  solutions  depicted,  showing  that  more  investment  can  lead  to  smaller  variations  in 
system  objectives.  The  third  and  most  interesting  observation  is  that  if  the  designer  is 
willing  to  invest  more  it  is  possible  to  further  reduce  objective  function  variations 
(through  adjusting  the  nominal  values  of  parameters)  than  that  achieved  through 
uncertainty  reduction  alone.  This  result  is  demonstrated  by  the  solutions  with  an  /Rvalue 
being  less  than  0. 13  in  Figure  5. 15,  or  the  entire  set  of  fi  solutions  corresponding  to  et3*  in 
Figure  5.14. 
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Figure  5,15:  Grinder  p  Solutions  for  a\* 
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As  can  be  seen  in  the  preceding  figures  there  is  actually  a  very  good  agreement 
between  the  two  approaches  for  finding  the  corresponding  /?*l a,*  solutions  suggesting 
again  that  the  DISA  approach  can  be  made  computationally  efficient  if  legacy  data  from 
in  the  first  stage  uncertainty  reduction  problem  is  used  to  solve  the  subsequent  design 
adjustment  problem.  To  further  demonstrate  this  fact  and  to  describe  a  sample  of  the 
improvements  to  the  nominal  design  suggested  by  the  DISA  approach  in  the  case  of  the 
angle  grinder.  Table  5.3  is  provided  which  contains  a  comparison  of  a  single  DISA  /?*l a,* 
solution  obtained  through  two  different  methods  (the  solution  point  called  out  on 

Figure  5.15  with  black  and  grey  crosses,  for  the  analysis  function  and  kriging  meta-model 
produced  solutions,  respectively). 

Table  5.3:  Sample  Grinder  Results  Comparison 


First  Stage  Analysis  Kriging 

(eg*)  Functions  Metamodel 


Rf 

0.138 

0.129 

0.129 

i 

0.165 

0.261 

0.261 

Rg 

0.394 

-0.353 

-0.354 

a 

P Analysis 

P Kriging 

Pi  ( Ro ) 

0.919 

-0.279 

0.292 

P2  (6) 

0.103 

-0.315 

-0.136 

Pi  1  ^Gap ) 

0.894 

-0.718 

-0.717 

PA  ( Lstack ) 

0.918 

0.514 

0.410 

P5  (Aw) 

0.889 

-0.189 

-0.508 

P6 C Rw) 

0.960 

-0.294 

0.248 

Table  5.3  also  contains  the  corresponding  Rf,  l  and  Rg  values  of  the  first  stage 
solution  for  comparison  purposes.  Note  that  the  Rg  value  following  the  first  stage 
procedure  is  greater  than  0,  indicating  that  the  determined  level  of  uncertainty  reduction 
is  incapable  of  guaranteeing  feasibility  which  drives  the  need  for  the  second  stage 
procedure.  Furthermore,  Table  5.3  suggests  that  for  this  specific  solution  the  copper  wire 
area  Aw  needs  only  a  small  uncertainty  reduction  of  11%  (a5  =  0.889)  and  an  equally 
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small  nominal  value  adjustment  of  less  than  2%  (ft?,  =  -0.189,  or  0.189x10%  ~  less  than 
2%  of  its  nominal  value  in  accordance  with  Tables  5.1  and  Table  5.2),  which  suggests 
that  the  uncertainty  in  this  parameter  does  have  a  large  effect  on  the  candidate  design’s 
performance  and  feasibility.  Conversely  this  DISA  solution  suggests  that  the  uncertainty 
in  the  mass  of  the  stator  thickness  (ts)  is  far  too  uncertain  {0.2  =  0.103),  and  additionally 
needs  to  have  its  nominal  value  reduced  as  well  by  (fi2  =  -0.315).  This  suggests  that  the 
uncertainty  in  the  stator  thickness  parameter  is  far  more  important  to  the  overall  design 
and  should  be  addressed  more  carefully.  These  conclusions  make  sense  given  that  the 
original  design  objectives  are  to  maximize  grinder  power  and  minimize  grinder  weight. 
Better  control  of  the  critical  parameters  of  the  grinder’s  motor  design  will  clearly  impact 
the  overall  grinder  power  in  a  positive  fashion.  Keep  in  mind  that  this  analysis  focuses 
on  just  one  potential  solution  for  this  problem  and  the  DISA  approach  produces 
numerous  options  just  like  this  one  for  the  designer  to  consider. 

In  terms  of  computational  effort  the  first  stage  a  problem  required  approximately 
2,640,000  function  calls.  An  additional  931  function  calls  were  required  following  the 
first  stage  procedure  in  order  to  sample  regions  of  the  parameter  space  not  adequately 
sampled  during  the  first  stage  optimization,  as  discussed  in  Section  5.3.  Each  of  the  p 
problems  solved  using  the  framework  shown  in  Figure  5.5(a)  required  additional 
3,400,000  function  calls,  on  average.  As  before,  solving  the  second  stage  problem  using 
the  formulation  shown  in  Figure  5.5(b)  required  practically  no  additional  function  calls 
(save  for  the  931  extra  calls  required  for  meta-model  refinement  prior  to  beginning  any 
second  stage  optimizations),  which  demonstrates  the  capability  of  the  meta-model 
assisted  DISA  approach  to  produce  a  rather  large  total  savings  in  function  calls. 
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5.5  SUMMARY  OF  RESEARCH  THRUST  3 


A  new  sensitivity  analysis  based  approach  using  a  sequential  two  stage  approach, 
DISA,  has  been  presented  for  use  in  managing  uncertainty  in  engineering  design.  For  a 
candidate  design  containing  interval  parameter  uncertainty  this  approach  has  the  ability  to 
determine  the  optimal  combination  of  uncertainty  reduction  in  the  first  stage 
optimization.  If  necessary,  the  DISA  approach  is  then  capable  of  determining  the  optimal 
combination  of  design  adjustments  required  to  minimize  system  objective  function 
variations  while  simultaneously  ensuring  feasibility  under  the  system’s  retained 
uncertainty  in  the  second  stage  optimization  problem.  Most  importantly,  by  using  a  two- 
stage  optimization  framework,  this  approach  arrives  at  those  solutions  without  searching 
the  entire  design  space  of  the  problem  and  in  the  process  ensures  that  new  design 
alternatives  are  as  similar  to  the  candidate  design  as  possible. 

This  approach  has  been  shown  to  be  effective  on  engineering  problems  of  varying 
difficulty  and  additionally  has  been  shown  to  be  capable  of  doing  so  rather  efficiently 
using  a  meta-model  assisted  framework.  It  was  shown  in  both  examples  that  given 
various  levels  of  optimal  uncertainty  reduction  this  new  DISA  approach  is  capable  of 
taking  an  infeasible  design  and  moving  that  design  away  from  the  nominally  active 
constraint  surfaces  by  a  small  amount  in  order  to  ensure  feasibility  while  maintaining 
performance  effectively.  In  the  more  complex  engineering  example  a  specific  study  was 
performed  on  a  single  candidate  solution  produced  by  the  DISA  approach  to  demonstrate 
the  ability  of  the  new  method  for  showing  a  designer  which  specifics  elements  of  an 
engineering  system  are  the  most  critical  to  both  the  feasibility  and  performance  of  the 
design.  More  importantly,  it  was  also  shown  that  the  DISA  approach  also  is  capable  of 
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suggesting  to  a  designer  how  best  to  use  available  investment  through  uncertainty 
reduction  and/or  small  design  adjustments. 

The  DISA  approach,  like  the  other  approaches  presented  in  this  dissertation  (and 
the  MOSA  approach  which  it  was  extended  from),  does  require  a  significant  level  of  a 
priori  information  to  employ.  The  DISA  approach  is  only  applicable  when  a  candidate 
design  already  exists  and  then  assumes  that  the  uncertainty  in  that  candidate  design  can 
be  reduced  about  the  nominal  parameter  values  for  the  candidate  design.  There  may  be 
many  design  problems  for  which  a  nominal  or  candidate  design  does  not  yet  exist,  but  for 
which  a  designer  must  have  a  means  for  considering  numerous  sources  of  reducible 
uncertainty.  The  next  chapter  of  this  dissertation  attempts  to  address  this  type  of  problem 
and  thus  provide  a  more  general  approach  for  the  design  of  engineering  systems  under 
reducible  interval  uncertainty. 
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CHAPTER  6:  REDUCIBLE  UNCERTAIN  INTERVAL  DESIGN 

(RUID) 

6.1  INTRODUCTION  TO  RESEARCH  THRUST  4 

In  the  previous  three  chapters  new  sensitivity  analysis  based  approaches  for  the 
design  of  systems  under  reducible  interval  uncertainty  were  presented.  These  approaches 
have  been  shown  to  be  effective  at  improving  the  performance  of  a  preexisting  candidate 
design  by  making  changes  to  the  known  nominal  uncertain  intervals  and/or  making  small 
adjustments  to  the  predetermined  nominal  parameter  values  themselves.  However,  the 
underlying  assumption  for  all  these  approaches  has  been  that  the  nominal  parameter 
values  and  nominal  uncertain  intervals  are  known,  which  may  often  not  be  the  case.  The 
work  presented  in  this  chapter  provides  designers  with  an  approach  for  the  design  of 
engineering  systems  under  reducible  interval  uncertainty  without  the  need  for  a 
preexisting  candidate  designs  and/or  known  uncertainty  levels,  thus  relaxing  the  limiting 
assumptions  required  by  the  previous  three  research  thrusts. 

Many  of  the  current  approaches  for  design  under  uncertainty  possess  limitations. 
Some  methods  require  large  and  difficult  to  obtain  data  requirements  for  statistical 
uncertainty  quantification  [Lui  et  al.,  2006;  Youn  and  Wang,  2008;  Benanzer  et  al., 
2009].  Other  methods  approximate  or  estimate  unknown  statistical  information  with 
evidence  theory  or  possibility  theory,  resulting  in  a  decrease  in  accuracy  when  compared 
to  probabilistic  methods  [Mourelatos  and  Zhou,  2006;  Zhou  and  Mourelatos,  2008].  Still 
other  methods  forgo  statistics  altogether  in  lieu  of  interval  analysis  but  require  designers 
assume  nominal  values  and  uncertain  intervals  for  the  uncertain  parameters  [Liao  and 
Chiou,  2008;  Hamel  et  al.,  2010;  Li  et  al.,  2009a].  All  of  these  approaches  have  distinct 
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strengths  and  weaknesses  that  a  designer  must  consider  when  determining  how  and  when 
to  consider  uncertainty  in  the  design  process.  Statistical  uncertainty  quantification  is 
certainly  desirable  but  requires  large  amounts  of  information  that  is  not  always  available. 
When  that  is  the  case,  should  a  designer  invest  in  obtaining  that  information?  Does  it 
always  make  sense  to  approximate  unknown  statistical  information  with  lower  fidelity 
estimations?  If  obtaining  statistical  information  is  not  feasible  or  desired,  what  uncertain 
intervals  should  the  designer  assume  or  estimate  for  uncertain  parameters?  Does  it  even 
make  sense  to  assume  a  nominal  value  for  an  uncertain  parameter  that  by  its  very  nature 
is  uncertain,  as  required  by  numerous  current  approaches  (e.g.  [Liao  and  Chiou,  2008; 
Hamel  et  al.,  2010;  Li  et  al.,  2009a])?  These  are  the  types  of  questions  that  a  designer 
must  be  able  to  answer  when  deciding  how  to  consider  reducible  sources  of  input 
uncertainty  and  the  answers  are  not  always  clear.  Furthermore,  these  questions  may  be 
particularly  difficult  to  answer  early  in  the  design  process  when  very  little  (if  any) 
information  is  available  about  the  types  and  sources  of  uncertainty  that  may  exist  within 
the  system  being  designed. 

When  dealing  with  irreducible  uncertainty  the  sources  uncertainty  becomes  a 
known  limiting  factor  in  the  system  and  must  simply  be  designed  around  (e.g.  robust 
optimization  as  discussed  in  Chapter  1).  However,  when  faced  with  reducible  uncertainty 
the  problem  becomes  more  complicated,  as  previously  stated.  A  designer  must  evaluate 
how  much  uncertainty  is  acceptable  and  must  consider  if  the  system’ s  performance  could 
be  improved  through  costly  investments  such  as  more  information,  better  equipment  or 
tighter  control  of  tolerances;  all  of  which  will  require  additional  resources  or  capital. 
RBDO  approaches  have  been  developed  that  allow  for  the  inclusion  of  additional 
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statistical  information  should  it  become  available  as  a  means  for  considering  reducible 
uncertainty  [Gunawan  and  Papalambros,  2006;  Youn  and  Wang,  2008],  Other 
approaches  combine  uncertainty  reduction  mechanisms  with  RBDO  techniques,  either 
through  sequentially  performing  RBDO  followed  by  uncertainty  reduction  [Qu  et  al., 
2003],  or  through  simultaneously  considering  reliability  and  uncertainty  reduction 
mechanisms  during  the  design  process  [Kale  and  Haftka,  2008].  There  are  also 
approaches  presented  in  the  literature  that  suggest  treating  the  variance  in  an  uncertain 
parameter  as  a  factor  to  control  within  an  RBDO  algorithm  provided  a  designer  has  a 
means  for  producing  lower  variance  in  input  parameters  [Benanzer  et  al.,  2009].  For  the 
cases  where  statistical  uncertainty  quantification  is  not  possible  some  recent  approaches 
have  suggested  that  the  uncertain  intervals  for  input  parameters  can  be  reduced  around  a 
known  nominal  value  (in  effect  specifying  tighter  tolerances)  in  order  to  improve  the 
performance  of  a  design  [Li  et  al.,  2009a;  Li  et  al.,  2009b,  Hamel  et  al.,  2010]. 

All  of  the  approaches  discussed  in  the  previous  paragraph,  and  in  Chapters  3-5  for 
that  matter,  are  extremely  effective  when  the  appropriate  information  is  available  with 
which  to  quantify  or  describe  any  relevant  uncertainty.  Collecting  additional  statistical 
information  or  specifying  tighter  tolerances  about  a  known  nominal  parameter  value 
necessarily  implies  that  the  uncertainty  in  that  parameter  is  known  to  some  extent,  either 
in  the  form  of  distributions  or  uncertain  intervals  about  that  known  nominal  value,  which 
may  not  always  be  the  case.  There  may  often  be  situations  where  a  system  designer 
faced  with  reducible  sources  of  uncertainty  will  know,  based  on  his/her  experience,  that 
the  input  parameters  of  a  design  will  possess  tolerances  and/or  uncertainties,  but  may 
have  no  way  of  estimating  or  specifying  those  tolerances  without  making  many  needless, 
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arbitrary  or  perhaps  even  erroneous  assumptions,  especially  at  early  stages  in  the  design 
process  when  information  is  necessarily  limited. 

Currently  there  is  no  existing  approach  capable  of  including  reducible  input 
uncertainty  within  an  optimization  framework  and  able  to  consider  the  following  factors: 
1)  multiple  sources  of  controllable  (or  reducible)  input  interval  uncertainty,  2)  limited 
resources  for  controlling  uncertainty,  3)  computationally  expensive  simulation  models,  4) 
no  available  statistical  information  about  uncertainty,  5)  unknown  nominal  values  for 
uncertain  parameters,  and  6)  no  reliable  means  for  estimating  uncertain  intervals.  This 
chapter  presents  a  new  multi-objective  optimization  approach  designed  to  meet  these 
needs.  This  new  approach,  called  Reducible  Uncertain  Interval  Design  (RUID),  is 
inspired  by  SA  approaches  in  that  it  relates  input  uncertainty  to  output  variations,  but 
does  not  require  a  designer  to  have  or  estimate  any  statistical  information,  uncertain 
intervals  or  nominal  values  for  any  uncertain  parameters.  The  only  values  a  designer 
must  specify  to  use  this  approach  are  the  extreme  upper  and  lower  bounds  on  any 
uncertain  input  parameters  for  which  the  uncertainty  is  reducible  or  controllable.  The 
RUID  approach  is  able  to  determine  the  optimal  uncertain  intervals  (upper  and  lower 
bounds)  for  the  reducible  uncertain  input  parameters  to  a  system  that  will:  i)  guarantee 
minimum  variations  in  system  outputs  under  uncertainty,  ii)  guarantee  minimal  required 
control  over  reducible  uncertainty  (recognizing  that  reducing  uncertainty  has  an 
associated  cost),  iii)  ensure  that  the  system’s  feasibility  is  satisfied  under  uncertainty,  and 
iv)  produce  design  solutions  with  as  close  to  the  deterministic  multi-objective  optimal 
system  performance  as  possible. 
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Simply  put,  the  RUID  approach  provides  designers  with  optimal  uncertain 
intervals  for  all  input  parameters  which  possess  reducible  uncertainty.  These  intervals  can 
then  be  used  to  specify  tolerances,  select  manufacturing  processes,  purchase  equipment 
and/or  make  any  other  uncertainty  related  design  decision  later  in  the  design  process. 
Figure  6. 1  depicts  the  manner  in  which  the  optimal  uncertain  intervals  produced  by  the 
RUID  approach  can  be  used.  An  uncertain  interval,  such  as  the  one  shown  at  the  top  part 
of  Figure  6.1,  will  be  specified  for  all  input  parameters  considered  by  the  RUID 
approach.  Once  those  intervals  are  determined,  as  long  as  a  designer  specifies  tolerances 
in  the  form  of  a  nominal  value  and  an  uncertain  interval  (left  side  of  the  lower  part  of  the 
figure)  or  selects  processes  with  statistical  uncertainty  (right  side  of  the  lower  part  of  the 
figure)  within  the  RUID  determined  upper  and  lower  bounds  for  all  reducible  uncertain 
parameters  considered,  the  performance  of  the  system  under  uncertainty  will  be 
guaranteed. 


RUID 


Lower 

Bound 


Upper 

Bound 


Parameter 

Value 


Figure  6.1:  Unknown  Uncertainty  Quantification 


RUID  makes  it  possible  to  perform  optimal  design  under  reducible  uncertainty 
without  the  need  for  limiting  or  degrading  assumptions  as  previously  discussed.  It  is 
formulated  to  handle  design  optimization  problems  possessing  multiple  objectives, 
multiple  constraints  and  mixed  continuous-discrete  input  parameters.  Additionally  the 
computational  efficiency  of  the  approach  is  improved  through  the  use  of  a  kriging  meta- 
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modeling  technique  applied  to  system  responses  obtained  early  in  the  approach.  This  new 
approach  does  require  an  increase  in  the  overall  dimensionality  of  the  problem  due  to  fact 
that  both  the  upper  and  lower  bounds  on  the  input  parameters  are  determined. 
Additionally,  while  the  use  of  meta-modeling  techniques  improves  the  efficiency  of  the 
approach,  this  improvement  does  create  the  potential  for  decreased  accuracy.  The 
strengths  and  weakness  of  this  new  approach  will  be  detailed  in  the  subsequent  sections. 
Section  6.2  contains  necessary  background  information  and  terminology.  The 
formulation  of  the  RUID  approach  is  presented  in  Section  6.3.  Section  6.4  presents  the 
results  obtained  using  the  RUID  approach  on  three  example  problems  of  varying 
difficulty.  Section  6.5  contains  some  concluding  remarks. 

6.2  BACKGROUND  AND  TERMINOLOGY 

This  section  contains  the  necessary  background  information  and  relevant 
terminology  for  the  RUID  approach  that  has  not  been  previously  presented  in  this 
dissertation. 

6.2.1  Input  Uncertainty  Level  Control 

The  set  of  uncertain  intervals  for  a  system  can  be  depicted  as  an  uncertain  region 
in  the  parameter  space  for  a  problem,  as  seen  in  previous  chapters  of  this  dissertation. 
Recall  that  the  term  “parameter  space”  describes  the  set  of  all  possible  input  parameter 
combinations  for  a  system  of  interest,  bounded  by  extreme  upper  and  lower  bounds  pub 
and  pib.  Figure  6.2  below  depicts  a  two  dimensional  visualization  of  a  general  parameter 
space  for  a  problem  comprised  of  two  inputs  p\  and  p2.  Shown  within  the  parameter  space 
in  Figure  6.2  is  an  input  uncertainty  region  defined  by  the  upper  and  lower  bounds  on 
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both  parameters  pv  and  pL  (or  the  uncertain  interval  [pi ,  pv\)  that  a  designer  would  like 
to  determine.  This  figure  is  very  similar  to  Figure  2.2  with  the  key  difference  that  no 
nominal  parameter  value  is  depicted  in  Figure  6.2  since  the  assumption  in  this  chapter  is 
that  nominal  parameter  values  are  unknown.  Instead  this  figure  depicts  an  uncertain 
region  within  the  parameter  space  determined  only  by  upper  and  lower  bounds,/;/*  <pi  < 
PV  ^  Pub- 

Input  Uncertainty 


Figure  6.2:  Input  Uncertainty  Region 

As  can  be  seen  in  Figure  6.2,  the  difference  between  the  size  of  the  input 
uncertainty  region  (the  dark  grey  rectangle  in  Figure  6.2)  and  the  region  depicting  all 
possible  parameter  combinations  (light  grey  rectangle)  can  be  thought  of  as  the  overall 
level  of  control  over  the  reducible  uncertain  parameters  in  the  system.  The  greater  the 
difference  in  size  between  the  two  regions,  the  lower  the  level  of  input  uncertainty  the 
system  will  possess.  Recall  that  the  RUID  approach  will  use  optimization  to  select  the 
upper  (pu)  and  lower  (p/J  bounds  for  all  input  parameters  to  a  system  that  possess 
reducible  uncertainty.  Eqn.  (6.1),  which  is  a  function  of  pu  and  p^,  provides  a 
formulation  for  describing  the  overall  level  of  control  over  the  reducible  uncertainty  in  a 
system. 
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Pu,k  ~  PL„k 
Pub.k  ~  Plb.k 


(6.1) 


W1  +  W2  ~  1 


This  formulation  is  adapted  from  the  Investment  metric  used  in  previous  chapters 
and  by  Li  et  al.  [2009a],  but  has  eliminated  the  need  for  nominal  parameter  values  and 
further  penalizes  extremely  low  uncertainty  levels  through  the  use  of  the  exponential 
decay  function.  The  metric  Ip  describes  the  overall  control  level  over  k  =  1  uncertain 
parameters,  where  A pk  represents  the  uncertain  interval  in  a  single  parameter  normalized 
by  the  constant  extreme  upper  and  lower  bounds  on  that  parameter.  As  the  elements  of 
A p  approach  zero  when  system  uncertainty  is  decreased,  the  value  for  Ip  will  approach  1 . 
High  Ip  indicates  the  need  for  greater  control  over  uncertain  parameters  (e.g.  lower 
tolerances  or  more  uncertainty  reduction),  which  will  necessarily  require  a  greater 
investment  in  resources  and/or  capital.  The  C  in  Eqn.  (6.1)  is  a  constant  that  can  be 
selected  by  the  designer  in  order  to  tune  the  relationship  between  uncertainty  levels  and 
the  costs  associated  with  reducing  system  uncertainty.  The  larger  the  value  for  C  the 
more  sensitive  the  value  for  Ip  is  to  small  changes  in  the  values  for  A When  solving 
Eqn.  (6.1)  for  Ip  throughout  this  chapter  the  constant  C  is  set  to  a  value  of  5.  This  value 
was  selected  based  on  empirical  observations  made  during  the  development  of  this  new 
approach  and  can  obviously  be  adjusted  as  necessary  to  fit  the  needs  of  a  particular 
design  problem.  Additionally  a  value  of  0.5  is  used  for  both  w\  and  w-2  throughout  this 
chapter,  again  based  on  empirical  observations. 
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6.2.2  Worst  Case  Uncertainty  Propagation 


Uncertainty  in  the  inputs  to  a  system  leads  to  a  corresponding  uncertainty,  or 
variation,  in  the  system’s  outputs.  In  the  case  of  a  system  that  a  designer  would  like  to 
optimize  using  Eqn.  (2.2),  uncertainty  in  p  will  lead  to  some  level  of  variation  in  both/ 
and  g  as  shown  in  Figure  6.3.  For  visualization  purposes  Figure  6.3  depicts  a  notional 
system  consisting  of  two  uncertain  parameters  in  the  parameter  space,  the  resulting 
variation  of  two  objective  functions  in  the  objective  or  performance  space,  and  the 
variation  in  two  constraint  functions  shown  in  the  constraint  space.  As  with  the 
parameter  space  defined  previously,  the  objective  and  constraint  spaces  describe  the  set 
of  all  possible  objective  and  constraint  function  combinations  possible,  respectively,  for  a 
particular  level  of  input  uncertainty. 

Regardless  of  how  a  system’s  input  uncertainty  is  quantified  it  is  essential  to 
understand  the  effect  of  that  uncertainty  on  the  performance  of  the  system.  If  the  input 
uncertainty  to  a  system  is  quanified  with  upper  and  lower  bounds,  one  means  for 
quantifying  the  effect  of  that  uncertainty  is  to  search  for  the  minimum  and/or  maximum 
values  of  all  output  functions  of  interest  (i.e.  objectives  and  constraints)  using  a  simple 
single  objective  optimization  procedure  for  each  maximum  and/or  minimum  output 
value.  For  example,  if  a  designer  is  interested  in  the  maximum  possible  value  for  an 
objective  function  (or  the  worst  case  value  if  the  objective  is  to  minimize  function/)  then 
the  optimization  problem  given  in  Eqn.  (6.2)  should  be  solved.  In  Eqn.  (6.2)  the  vector  pv 
is  again  used,  which  represents  a  single  realization  of  the  uncertain  values  of  the  vector  p 
bounded  by  pu  and  p l.  This  vector  is  again  used  as  the  decision  variables  when 
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attempting  to  quantify  the  effects  of  a  set  of  known  input  uncertain  intervals  on  a 
system’s  outputs. 


/m-  =  max  f,(py) 

P , 

where  pL<  pr  <  pv 


(6.2) 


If  the  minimum  value  for  an  objective  function  (or  best  possible  performance)  is 
desired,  then  the  optimization  problem  in  Eqn.  (6.3)  must  be  solved. 


/.  =  min  f,(p  ) 

J  t,mm  „  J  I  v  i  t  ' 

p . 

where  pL<  pr<  pu 


(6.3) 


Lastly,  if  a  designer  is  interested  in  the  feasibility  of  a  system,  then  it  is  important 
to  know  if  any  of  the  constraint  functions  will  potentially  have  a  value  greater  than  0.  As 
such,  in  order  to  check  the  feasibility  of  a  system  under  uncertainty  it  is  necessary  to 
check  the  maximum  possible  values  for  all  constraint  functions.  This  can  potentially  be 
accomplished  by  solving  Eqn.  (6.4)  for  each  constraint  function. 


8j .«  =  8j(P,) 

where  pL  <  pr  <  pv 


(6.4) 


Figure  6.3:  Uncertainty  Propagation 
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If  a  designer  is  attempting  to  solve  the  multi-objective  optimization  problem 
given  in  Eqn.  (2.2)  for  a  system  known  to  possess  input  parameter  uncertainty,  the  worst 
case  variation  in  the  system’s  objective  functions / can  be  quantified  with  a  single  value 
as  shown  in  Eqn.  (6.5). 


=  llv 


where 


A/,  = 


/, 


/, 


i,good 


f, 


bad 


(6.5) 


In  the  above  equation  fjnax  and  fljnm  are  found  using  Eqn.  (6.2)  and  Eqn.  (6.3) 
respectively.  Recall  that  solving  Eqns.  (6.2)  and  (6.3)  for  fimax  and  fijnin  requires  that  the 
values  for  pv  and pL  be  specified.  The  top  half  of  Figure  6.4  graphically  shows  V/for  a  bi¬ 
objective  system.  The  denominator  of  each  A fi  value  in  Eqn.  (6.5)  consists  of  two 
constant  normalization  factors  figood  and  fibad  which  must  be  determined  prior  to 
evaluating  Eqn.  (6.5).  These  normalization  or  scaling  factors  should  be  selected  to  ensure 
that  each  A f  value  considered  is  between  0  and  1,  with  0  describing  no  variation  and  1 
describing  maximum  variation  in  the  z'th  system  objective  given  a  specified  level  of 
system  uncertainty.  This  scaling  makes  it  possible  to  compare  multiple  A/]  values  of 
drastically  different  orders  of  magnitude  using  a  II. IU  as  given  in  Eqn.  (6.5).  The  method 
for  determining  these  scaling  factors  within  the  context  of  the  RUID  approach  will  be 
discussed  in  Section  6.3. 


As  stated  previously,  variation  in  p  may  very  well  lead  to  system  failure,  or 
infeasibility.  Recall  that  in  Eqn.  (2.2)  infeasibility  results  when  at  least  one  of  the 
system’s  constraint  functions  has  a  value  greater  than  0.  In  order  to  check  this,  given  a 
specified  level  of  system  input  uncertainty,  Eqn.  (6.6)  can  be  evaluated. 
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(6.6) 


V  =max(  max  gXp,)) 

*  /%  i  J 

where  pL<  pt<,  pu 

As  with  Eqn.  (6.5),  Eqn.  (6.6)  is  solved  for  specified  values  of  pu  and  p/.  If  the 
value  for  Vg  is  greater  than  0  the  system  may  become  infeasible  given  the  specified  input 
uncertainty  intervals.  This  formulation  is  used  to  find  Vg  in  a  single  step  in  order  to  avoid 
the  computational  effort  that  could  result  from  considering  each  constraint  function 
independently  through  the  use  of  Eqn.  (6.4). 

The  bottom  half  of  Figure  6.4  graphically  depicts  Vg  for  a  two  constraint  system 
for  two  different  levels  of  input  uncertainty.  In  Figure  6.4  the  dark  grey  uncertainty 
interval  will  result  in  a  Vg  value  greater  than  0  and  an  infeasible  system.  Also  shown  in 
Figure  6.4  is  the  correlation  between  Ip,  or  input  uncertainty  control,  and  Vf  and  Vg.  If  a 
designer  can  afford  to  place  greater  control  over  system  uncertainty  the  most  likely 
outcome  will  be  lower  variations  in  system  outputs  and  in  some  cases  will  eliminate  the 
potential  for  system  failure. 


/2 


Figure  6.4:  Propagated  Uncertainty  Comparison 
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6.2.3  Anchor  points 


If  all  the  elements  of  the  vector  p  are  known  with  certainty  (i.e.,  p  =  pu  =  Pl )  then 
Eqn.  (2.2)  can  be  solved  in  a  deterministic  sense  for  the  optimal  values  for  p.  The  set  of 
Pareto  optimal  solutions  that  are  produced  describe  the  best  performance  attainable  for 
the  system  in  a  multi-objective  sense.  Each  of  the  determined  Pareto  optimal  solutions 
corresponds  to  a  different  deterministic  solution  for  the  elements  of  the  vector  p, 
assuming  the  values  for  p  are  known  with  certainty.  However,  as  previously  stated 
uncertainty  in  the  input  parameters  for  most  systems  is  unavoidable.  As  a  result,  the 
values  for  p  may  very  well  possess  some  uncertainty  and  thus  the  performance  of  the 
obtained  deterministic  Pareto  optimal  solutions  cannot  be  guaranteed.  Figure  6.5  depicts 
this  likely  scenario  in  the  parameter  space  for  a  two  parameter  system  given  a  single 
objective  function  contour  for  simplicity. 

P2 


Figure  6,5:  Variation  from  Deterministic  Optima 

As  shown  in  Figure  6.5,  for  some  realizations  of  p  the  performance  of  the  system 
may  improve  (f  may  decrease),  while  for  other  realizations  the  performance  may  degrade 
and  the  designer  cannot  reliably  know  which  of  the  two  will  occur.  This  relationship 
between  input  uncertainty  and  output  variation  is  present  in  all  outputs,  including 
objectives  and  constraints,  further  exacerbating  the  issue.  When  optimizing  a  system 
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under  uncertainty  a  designer  must  decide  what  realizations  of  p  to  focus  on  in  the 
optimization  problem  as  a  result  of  these  potential  variations.  As  previously  discussed,  a 
designer  of  a  system  that  possesses  input  uncertainty  could  optimize  the  expected  value 
of  the  system's  outputs  in  a  statistical  sense  if  data  is  available,  or  if  data  is  not  available 
the  designer  could  optimize  the  system  around  an  assumed  nominal  design.  In  either  case 
the  designer  must  make  an  assumption  about  the  system  and  in  extreme  cases  the  system 
may  perform  very  different  from  the  expected  or  nominal  performance,  especially  in 
multi-objective  problems. 


h 


Anchor  Point  Set  fA  =  fipA) 


*/l 


Figure  6.6:  RUID  Anchor  Points 


An  alternate  approach  to  the  challenge  of  optimizing  a  design  under  uncertainty  is 
to  treat  the  deterministic  Pareto  optimal  solution  set  for  a  system  of  interest  as  the  ideal 
performance  of  the  system.  Using  this  approach  a  designer  would  like  the  system’s 
performance  under  uncertainty  to  vary  as  little  as  possible  from  the  deterministic  optimal 
performance.  To  this  end  a  Pareto  optimal  set  of  solutions  for  a  system  of  interest, 
determined  by  solving  Eqn.  (2.2)  for  p  in  a  deterministic  sense,  can  be  treated  as  target 
values,  or  anchor  points,  for  a  follow  on  optimization  under  uncertainty  procedure,  as 
shown  in  Figure  6.6. 
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6.2.4  Anchor  Point  Upper  and  Lower  Bounds 


Once  a  set  of  M  anchor  points  p a  is  obtained,  as  detailed  in  the  previous  section, 
the  uncertainty  associated  with  the  m,h  anchor  point  (pAm))  can  be  described  by  the 
distance  from  the  anchor  point  to  the  upper/lower  bounds  on  all  uncertain  parameters,  as 
shown  in  Eqn.  (6.7). 


Pv=PiA)+du 

PL=PT~dL 


(6.7) 


The  vectors  dv  and  dL  have  the  same  number  of  elements  as  pA  and  describe  the 
uncertainty  associated  with  the  mth  anchor  point  as  shown  in  Figure  6.7.  The  values  for 
dv  and  dL  should  be  limited  to  values  between  0  and  a  designer  specified  maximum 
distance,  which  could  typically  be  some  percentage  of  the  difference  between  the  extreme 
upper  and  lower  bounds  (p„b  and  pw)  on  the  system’s  inputs.  The  index  of  the  anchor 
point  m  and  the  anchor  point  distances  dv  and  dL  associated  with  that  anchor  point  will  be 
selected  systematically  by  the  RUID  approach  thus  defining  the  upper  and  lower  bounds 
on  the  reducible  uncertain  parameters  of  a  system. 


Pi 


Figure  6.7:  Uncertain  Region  for  the  m"'  Anchor  Point 
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6.3  REDUCIBLE  UNCERTAIN  INTERVAL  DESIGN  (RUID) 


As  previously  stated  the  Reducible  Uncertain  Interval  Design  (RUID)  approach 
determines  the  optimal  upper  and  lower  bounds  for  input  parameters  to  a  system  that 
possesses  reducible  uncertainty.  The  resulting  solutions  will  produce  minimal  variation 
from  the  deterministic  Pareto  optimal  performance  of  the  system  provided  a  maximum 
level  of  input  uncertainty  in  a  multi-objective  sense.  This  is  accomplished  by  solving  two 
different  multi-objective  optimization  problems  in  sequence.  First  the  system  is 
optimized  in  a  deterministic  sense  with  respect  to  the  engineering  objectives  /  and  the 
parameters  of  interest  that  possess  reducible  uncertainty  p  in  order  to  provide  a  set  of 
anchor  points.  Then  a  second  bi-objective  optimization  problem  is  solved  that  seeks  to 
maximize  the  reducible  uncertainty  in  the  system’s  inputs  (minimum  Ip  value  as  given  in 
Eqn.  (6.1))  provided  the  solution  contains  a  deterministic  anchor  point,  while 
simultaneously  minimizing  the  variations  of  the  systems  outputs.  The  computational 
efficiency  of  the  approach  is  improved  through  the  use  of  kriging  meta-models  built 
using  function  call  data  obtained  during  the  deterministic  optimization  procedure  and 
refined  as  necessary  during  the  second  optimization  under  uncertainty  procedure. 

6.3.1  RUID  Formulation 

As  previously  stated,  the  RUID  approach  involves  sequentially  solving  two 
optimization  problems.  The  first  stage  optimization  problem  optimizes  the  performance 
of  the  engineering  system  with  respect  to  the  uncertain  parameters  in  a  deterministic 
sense,  thus  providing  a  set  of  anchor  points  for  use  in  the  second  stage  of  the  approach. 
This  is  accomplished  by  solving  Eqn.  (2.2).  The  first  stage  optimization  problem  also 
generates  a  large  amount  of  system  information  (f(pv)  and  g(pv)  data)  which  is  retained 
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and  used  to  generate  system  output  function  scaling  factors  and  kriging  meta-models  of 
the  system  output  functions  for  use  in  the  second  stage  optimization  problem.  The 
second  stage  problem  then  focuses  on  solving  the  optimization  problem  given  in  Eqn. 
(6.8). 

min  Vf 
min  I 

m,  dv  ,dL  p 

s.t. 

Vg<0 

Vf  <8  or  I p  <  e  ( optional )  (6.8) 

where 
me  [1 
0  <  dv,dL 

The  decision  variables  in  Eqn.  (6.8)  are  the  index  of  the  anchor  point  of  interest 
m,  along  with  the  vectors  dv  and  d L  which  describe  the  upper  and  lower  bounds  of  the 
input  uncertainty  with  respect  to  the  mth  anchor  point.  When  a  set  of  M  anchor  points  are 
known,  the  index  m  can  be  used  to  look  up  the  values  for  Pa"1)  and  then  Eqn.  (6.7)  can  be 
used  to  determine  the  upper  and  lower  bounds  pv  and  pL  on  all  reducible  input  parameters 
given  dv  and  dL.  This  obviously  represents  an  increase  in  the  dimensionality  of  the 
original  problem,  but  that  increase  allows  for  an  extremely  low  number  of  a  priori 
assumptions  as  listed  in  Section  6.1.  Recall  that  Vf  is  determined  by  solving  Eqn.  (6.5) 
given  pv  and  pL.  Ip  also  depends  on  pv  and  pL  and  is  determined  by  solving  Eqn.  (6.1). 
Lastly  Vg  is  determined  by  solving  Eqn.  (6.6),  again  given  pv  and  pL.  The  other  two 
constraints  listed  in  Eqn.  (6.8)  are  optional  and  simply  allow  the  designer  to  specify 
boundaries  on  Vf  and/or  Ip  if  desired.  It  is  important  to  note  that  solving  Eqns.  (6.5)  and 
(6.6)  both  involve  solving  single  objective  optimization  problems.  As  a  result,  Eqn.  (6.8) 
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is  a  two-level  nested  optimization  problem.  This  nested  structure  can  obviously  result  in 
high  computational  effort  as  is  the  case  for  many  design  optimization  approaches  that 
include  parameter  uncertainty  as  discussed  in  Section  6.1.  The  computational  effort  of  the 
RUID  approach  is  reduced  through  the  use  of  kriging  meta-models,  as  discussed  in 
Chapter  2,  built  using  the  data  retained  during  the  first  stage  optimization  problem.  This 
process  will  be  discussed  in  detail  in  the  following  sections. 

Eqn.  (6.8)  uses  a  bi-objective  optimization  framework  to  place  system  objective 
function  variations  (V/)  in  tension  with  the  cost  required  to  control  (or  reduce)  system 
input  uncertainty  (Ip).  These  two  competing  objectives  are  formulated  such  that  for  a 
given  decrease  in  output  uncertainty,  which  is  desirable,  there  is  an  associated  increase  in 
cost  to  the  designer,  which  is  unavoidable.  Additionally,  the  use  of  an  anchor  point  set 
and  the  vectors  du  and  dL  as  the  decision  variables  in  Eqn.  (6.8)  makes  it  possible  for  the 
RUID  approach  to  guarantee  that  any  uncertain  solution  produced  by  the  approach  (as 
defined  by  pu  and  pL)  will  be  as  close  to  a  deterministic  Pareto  optimal  design  for  the 
system  as  possible.  This  is  because  any  uncertain  solution  produced  by  Eqn.  (6.8)  will 
necessarily  contain  a  deterministic  Pareto  optimal  solution  within  the  bounds  of  the 
uncertain  intervals  of  that  solution. 

6.3.2  Steps  of  the  RUID  Approach 

Figure  6.8  contains  a  flow  chart  for  the  overall  RUID  algorithm. 
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Figure  6.8:  RUID  Approach 


The  steps  of  the  approach  are  as  follows: 

Stepl:  Solve  Eqn.  (2.2)  for  p*  (optimal  p)  in  a  deterministic  sense  (i.e.  no 
uncertainty).  During  this  initial  optimization  problem  retain  all  function  call  data. 


Step  2:  Deterministic  Pareto  optimal  solutions  obtained  provides  the  set  of  anchor 
points  Pa. 


Step  3:  Search  set  of  retained  function  call  data  for  the  maximum  and  minimum 
observed  values  for  each  of  the  f  objectives  functions.  Set  maximum  and  minimum 
observed  values  as  fi,g00d  and  f,bad,  respectively,  for  each  objective  function  to  serve  as 
scaling  factors  in  Eqn.  (6.5).  Recall  that  Eqn.  (6.6)  does  not  require  scaling  factors  for 
calculating  Vg,  so  scaling  data  for  constraint  functions  is  not  required. 

Step  4:  Using  retained  function  call  data,  build  kriging  meta-models  of  all  system 
objective  and  constraint  functions. 
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Step  5:  Solve  Eqn.  (6.8)  to  determine  Pareto  optimal  set  of  upper  and  lower 
bounds  on  all  input  parameters  pu  and  p Solving  Eqn.  (6.8)  requires  iteratively  varying 
the  selected  anchor  point  index  m  and  the  variables  dv  and  dv  and  then  solving  Eqns. 
(6.2),  (6.3)  and  (6.6)  to  obtain  Vf,  Ip  and  Vg.  This  process  is  called  the  inner  problem  and 
consists  of  solving  numerous  single  objective  optimization  problems  focused  on 
determining  the  maximum  variations  of  all  system  output  function  given  specified  levels 
of  input  uncertainty.  Figure  6.9  contains  a  flow  chart  of  the  inner  problem. 


Figure  6,9:  RUID  Inner  Problem 


The  steps  of  the  inner  problem  are  as  follows: 


Step  a:  Given  m,  dv  and  dL  from  optimizer,  look  up  pAm)  from  the  anchor  point 
set  and  evaluate  Eqn.  (6.7)  to  obtain  pu  and  p /_. 
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Step  b:  Solve  Eqns.  (6.2),  (6.3)  and  (6.6)  for  each  system  objective  and/or  the 
system  constraint  functions.  For  each  function  the  meta-model  built  during  Step  4  is  used 
in  lieu  of  the  actual  system  analysis  models. 

Step  c:  If  the  kriging  error  estimate  for  any  evaluation  is  too  high,  the  actual 
function  is  evaluated  to  ensure  accuracy  and  the  newly  obtained  function  call  is  retained 
to  refine  meta-model(s). 

Step  cl :  Rebuild  meta-model(s),  if  necessary. 

6.3.3  RUID  Computational  Efficiency 

The  unique  capability  of  the  kriging  interpolation  technique  makes  it  possible  for 
the  RUID  approach  to  be  more  efficient.  To  solve  a  multi-objective  optimization 
problem  in  a  deterministic  sense  using  MOGA  requires  G  generations  and  H  populations, 
resulting  in  a  computation  effort  of  the  order  O(GH).  Solving  a  multi-objective 
optimization  problem  that  includes  input  uncertainty  requires  the  use  of  an  inner  problem 
that  consists  of  solving  several  optimization  problems  for  each  population  point 
considered  in  each  generation.  In  the  case  of  the  RUID  approach  the  inner  problem 
requires  solving  an  additional  set  of  single  objective  maximization  problems,  one  for 
each  of  the  I  objective  functions  and  one  for  all  J  constraint  functions  considered 
together.  If  each  single  objective  inner  optimization  problem  requires  a  total  of  U 
function  calls,  the  resulting  computational  effort  for  RUID  would  be  of  the  order 
0{GH(2I+\)U).  Since  the  RUID  approach  involves  solving  both  a  deterministic  and 
uncertain  optimization  problem,  the  effort  required  by  RUID  in  the  absence  of  any 
surrogate  approximation  would  be  of  the  order  0(G\H\+G2H2(2I+1)U)  in  the  worst  case, 
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where  the  subscript  1  represents  the  deterministic  first  stage  problem,  while  the  subscript 
2  denotes  the  second  stage  optimization  problem. 

Clearly  this  level  of  computational  effort  could  quickly  become  unacceptable  for 
any  system  that  consists  any  more  than  a  few  objective  and/or  constraint  functions. 
However,  using  meta-models  built  from  the  function  call  data  generated  by  the  first  stage 
deterministic  optimization  procedure  a  significant  savings  is  possible,  even  considering 
the  additional  V  function  calls  required  by  the  inner  problem  when  the  kriging  predicted 
error  [Martin  and  Simpson,  2005]  is  too  high  (as  shown  in  Figure  6.9).  The  resulting 
meta-model  assisted  algorithm  would  have  a  computational  effort  of  the  order 
0{G\H\+V),  which  yields  a  total  savings  of  GiH4U+ 1  )U  -  V  function  calls.  In  this  work 
a  kriging  predicted  error  of  1%  of  the  function  value  was  considered  too  high,  but  this 
value  can  be  adjusted  by  a  designer  as  appropriate.  The  accuracy  of  this  technique  will 
be  demonstrated  in  the  next  section  for  each  of  the  example  problems. 

6.4  EXAMPLES  AND  RESULTS 

In  this  section  the  RUID  approach  is  applied  to  one  numerical  and  two 
engineering  example  problems  of  varying  complexity,  as  presented  in  the  first  three 
subsections.  The  accuracy  of  the  meta-modeling  strategy  used  will  be  addressed  in 
Section  6.4.4. 

6.4.1  Numerical  Example 

The  first  example  problem  presented  is  a  two-dimensional  constrained  multi¬ 
objective  problem  called  TNK  [Deb,  2001].  The  formulation  of  TNK  is  provided  in  Eqn. 
(6.9). 
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min/,  =  px 

P 

min  f2  =  p2 

P 


s.t. 


Si  =  {Pi  —  0-5)2  +  (77,  -  0.5)2  -  0.5 


g2  —  -p2  -  p\  + 1  +  O.lcos 
0<l\,p2  <n 


P 

16arctan— 


(6.9) 


Since  TNK  is  a  two  dimensional  problem  where  the  two  objectives  are  to 
minimize  the  two  decision  variables,  the  results  can  be  visualized  graphically.  RUID  was 
applied  to  the  TNK  problem  using  MOGA  as  the  optimizer.  A  population  size  of  40  and 
100  generations  were  used  in  the  first  stage  and  the  total  realized  computational  effort 
was  2,218  function  calls.  In  the  second  stage  optimization  a  population  size  of  80  and 
100  generations  was  used  and  required  only  4,657  additional  function  calls  (required 
when  the  kriging  predicted  error  was  too  high),  vice  the  maximum  of  40,000,000  calls 
that  could  have  been  required  if  meta-modeling  had  not  been  used.  This  maximum 
number  was  calculated  by  multiplying  the  number  of  generations  (G  =  100)  by  the 
population  size  ( H  =  80)  by  the  sum  of  2  times  the  number  of  objectives  (/  =  2)  plus  one 
(for  the  Vg  optimization)  times  the  number  of  function  calls  required  for  each  inner 
problem  optimization  on  average  (U  =  1000)  using  the  procedure  discussed  in  Section 
6.3.3. 

The  second  stage  optimization  produced  the  Pareto  solutions  provided  in  Figure 
6.8,  while  the  anchor  points  produced  by  the  first  stage  optimization  along  with  a  set  of 
four  representative  RUID  solutions  are  depicted  in  the  parameter  space  for  the  TNK 
problem  in  Figure  6.11.  The  Vf,  Ip  and  determined  upper  and  lower  bounds  for  the  four 
representative  solutions  are  provided  in  Table  6.1  and  are  called  out  in  Figure  6.10  with 
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grey  fill  for  comparison  purposes.  To  obtain  these  solutions  Vf  was  constrained  to  be  less 
than  0.1  (or  10%  maximum  variation  in  system  objective(s))  and  the  maximum  values  for 
da  and  dL  were  both  limited  to  10%  of  the  maximum  values  for  p  as  shown  in  Eqn.  (6.9). 


1.0 
0.9 
0.8 
0.7 
0.6 

/  0.5 

lP 

0.4 

0.3 

0.2 

0.1 

0.0 
C 

Figure  6.10:  RUID  Pareto  Solutions  for  TNK  Problem 
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Table  6.1:  Select  RUID  Pareto  Points  for  TNK  Problem 


Vf 

Ip 

Pu.i 

Pu 

PU.  2 

PL.  2 

a 

0.016 

0.807 

0.519 

0.495 
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0.843 

b 
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0.727 
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0.529 

c 
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0.65 1 

0.640 
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Figure  6.11:  TNK  Problem  Parameter  Space 

Figures  6.10  and  6.11  clearly  show  the  capability  of  the  RUID  approach  to 
provide  a  designer  with  a  set  of  design  alternatives  for  the  upper  and  lower  bounds  on  the 
system’s  two  inputs,  leaving  the  designer  with  the  freedom  to  choose  the  level  of 
acceptable  output  variations  based  on  required  input  uncertainty  control  (or  reduction). 
For  example,  solution  a  has  an  extremely  low  Vf  value,  but  achieves  that  low  output 
variation  through  tight  control  over  the  uncertainty  in  p\  and  p2,  as  demonstrated  by  the 
extremely  high  Ip  value  and  the  extremely  small  region  of  uncertainty  associated  with  that 
solution  in  Figure  6.11.  This  example  problem  is  a  difficult  bi-objective  optimization 
problem  and  shows  the  capability  of  the  RUID  approach  to  find  uncertain  solutions  that 
do  not  violate  any  constraints  while  simultaneously  achieving  as  close  to  deterministic 
Pareto  optimal  performance  as  possible  given  the  specified  level  of  input  uncertainty. 

6.4.2  Simple  Engineering  Design  Problem 

The  second  example  problem  presented  is  a  simple  two-dimensional  engineering 
optimization  problem  from  Arora  [2004],  previous  used  in  Chapter  5.  This  problem  was 
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chosen  again  in  order  to  again  present  the  obtained  uncertain  interval  results  graphically 
thus  clearly  demonstrating  the  capabilities  of  the  RUID  approach  further.  Figure  6.12 
depicts  the  system  graphically  while  Eqn.  (6.10)  formulates  the  corresponding 
optimization  problem. 
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Figure  6.12:  Tube  Design  Problem 


min  /,  =  IphnRt 

■  f  R 

mm  f ,  =  — 

*•>  t 

s.t. 


p.  = - a  <0 

1  2nRt 


g1  =  F- 


n3ER3t 


4  h1 

0.10  £  RZ  0.30 
0.02  <t  <0.10 


<0 


(6.10) 


Eqn.  (6.10)  was  first  solved  in  a  deterministic  sense  using  a  population  size  of  40 
and  100  generations  and  required  1,591  function  calls.  As  before  this  procedure 
produced  a  set  of  deterministic  Pareto  optimal  points,  which  again  served  as  the  anchor 
points  for  the  second  stage  of  the  RUID  approach.  Next,  Eqn.  (6.8)  was  solved  by 
MOGA  using  a  population  size  of  80  and  100  generations.  For  this  problem,  as  with  the 
previous  example,  the  maximum  possible  number  of  required  function  calls  that  would 
have  been  required  if  kriging  had  not  used  was  40,000,000.  In  reality  the  second  stage 
actually  only  required  an  additional  4,922  function  calls,  resulting  in  a  total  function  call 
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savings  of  almost  100%.  This  savings  was  calculated  by  subtracting  the  total  additional 
function  calls  required  from  the  maximum  total  possible,  and  then  dividing  that  number 
by  the  maximum  possible  function  calls.  The  population  size  was  increased  from  40  to  80 
in  the  second  stage  optimization  procedure  in  order  to  better  converge  given  the  increased 
dimensionality  of  the  second  stage  of  the  RUID  problem.  Recall  that  a  deterministic 
optimization  procedure  determines  the  values  for  each  of  the  elements  of  the  vector  p, 
while  the  second  stage  optimization  in  the  RUID  approach  solves  for  both  the  upper  and 
lower  bound  on  each  of  the  elements  of  the  vector  p.  As  before  Vj  was  again  limited  to  a 
maximum  of  0.1  and  for  this  problem  the  anchor  point  distance  vectors  (du  and  di.)  were 
varied  between  0  and  10%  of  the  difference  between  the  extreme  upper  and  lower  bounds 
for  R  and  t  provided  in  Eqn.  (6. 10). 

Figure  6.13  shows  the  Pareto  optimal  frontier  obtained  by  solving  Eqn.  (6.8)  for 
the  tube  design  problem  shown  in  Eqn.  (6.10).  Each  solution  on  the  Pareto  frontier 
depicts  an  optimal  option  for  the  upper  and  lower  bounds  on  the  two  input  parameter  for 
the  tube  design  problem,  the  radius  of  the  tube  R  and  the  thickness  of  the  tube  t.  The 
entire  Pareto  frontier  is  shown  on  Figure  6.13,  while  four  representative  solutions  have 
been  called  out  with  grey  fill  as  with  the  previous  example.  The  values  for  the  RUID 
determined  upper  and  lower  bounds  pu  and  Pl,  along  with  the  corresponding  Vf  and  Ip 
values  for  each  of  the  four  representative  solutions  are  provided  in  Table  6.2.  Lastly  the 
optimal  uncertain  regions  for  each  of  the  four  representative  solutions  have  been  plotted 
in  the  parameter  space  for  the  tube  design  problem  in  Figure  6.14,  along  with  the 
deterministic  Pareto  optimal  solutions  which  served  as  the  anchor  points  for  the  RUID 
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algorithm.  Figure  6.14  also  includes  constraint  function  contours  to  define  the  feasible 
domain  of  the  problem,  along  with  objective  function  contours  for  comparison  purposes. 


Figure  6.13:  RUID  Pareto  Solutions  for  Tube  Problem 
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Figure  6.14:  Tube  Problem  Parameter  Space 
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Solution  a  in  Table  6.2  and  Figure  6.13  and  6.14  has  extremely  low  variation  in 
the  system’s  objective  functions,  but  in  order  to  achieve  that  low  variation  tight  control 
over  the  input  parameters  is  necessary,  including  only  0.005  m  of  uncertainty  in  R  and 
only  0.002  m  of  uncertainty  in  t.  Solution  d  on  the  other  hand  has  significantly  more 
variation  in  the  objective  (higher  V/)  but  is  also  much  cheaper  to  obtain  in  that  it  has  a 
correspondingly  low  Ip  value.  If  the  designer  of  the  tube  system  is  happy  with  the 
variation  level  associated  with  solution  d,  then  that  design  can  be  achieved  with  much 
lower  tolerances  on  R  and  t.  However,  if  it  is  extremely  important  that  the  variation  in 
the  performance  of  the  tube  design  be  as  low  as  possible,  then  the  designer  can  select  the 
further  reduced  uncertainty  in  R  and  t  as  specified  by  solution  a.  The  RUID  approach 
provides  designers  with  a  suite  of  options  and  tradeoff,  such  as  those  discussed  above,  to 
aide  in  determining  input  parameter  tolerances  and  potential  uncertainty  reductions 
without  making  any  a  priori  assumptions  about  the  statistical,  expected  or  nominal  values 
of  the  parameters.  It  is  also  important  to  note  that  in  Figure  6.14  all  the  solutions 
depicted  contain  a  deterministic  optimal  anchor  point  within  the  uncertain  regions 
determined  by  the  RUID  approach  based  on  the  formulation  of  the  approach  and  thus 
guarantee  as  close  to  deterministic  Pareto  optimal  performance  as  possible  under 
uncertainty. 

6.4.3  Heat  Exchanger  Design  Problem 

In  order  to  demonstrate  the  RUID  approach  on  a  more  complicated  system,  a 
thermal-fluid  problem  taken  from  Magrab  et  al.  [2005],  was  adapted  into  the  bi-objective 
optimization  problem  given  in  Eqn.  (6.11)  and  depicted  graphically  in  Figure  6.15.  This 
problem  is  concerned  with  the  design  of  a  shell  and  tube  heat  exchanger  that  uses  a  flow 
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of  cold  water  to  remove  heat  from  a  flow  of  hot  water.  One  fluid  flows  through  a  set  of 
small  tubes  mounted  inside  a  larger  shell,  while  the  other  fluid  flows  over  the  smaller 
tubes  within  the  shell.  The  overall  system  objectives  are  to  maximize  the  heat  transfer 
rate  of  the  heat  exchanger  (a  measure  of  how  much  energy  is  moved  from  the  cold  fluid 
to  the  hot  fluid)  while  simultaneously  minimizing  the  length  of  the  heat  exchanger.  The 
system  consists  of  6  input  parameters  (including  1  discrete  parameter),  2  objectives  and 
multiple  thermal-fluid,  performance  and  geometric  constraints,  as  shown  in  Eqn.  (6. 1 1 ). 
For  details  on  this  system  see  Magrab  et  al.  [2005], 


Figure  6.15:  Shell  and  Tube  Heat  Exchanger  Problem 
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The  heat  exchanger  problem  was  solved  using  the  procedure  described  in  Section 
6.3  just  as  with  the  previous  two  example  problems.  The  Pareto  set  of  uncertain  solutions 
produced  are  shown  in  Figure  6.16  and  as  with  before  a  set  of  4  representative  solutions 
have  been  called  out  with  dark  grey  fill  and  the  details  for  these  4  solutions  are  shown  in 
Table  6.3.  As  with  the  previous  examples  both  the  first  stage  and  second  stage  problems 
were  solved  with  MOGA  using  100  generations  in  each  stage.  When  considering  both  the 
function  calls  required  by  the  first  stage  optimization  and  the  additional  function  calls 
required  by  the  inner  problem  of  the  second  stage  problem  for  the  cases  when  the  meta¬ 
model  predicted  error  was  too  high,  an  overall  computational  savings  of  about  100%  was 
again  observed  when  compared  to  the  worst  case  as  described  in  Section  6.3.3,  despite 
the  increased  dimensionality  and  increased  number  of  constraint  functions  in  this 
problem. 
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Figure  6,16:  RUID  Pareto  Solutions  for  Heat  Exchanger 
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Table  6.3:  Select  RUID  Pareto  Points  for  Heat  Exchanger  Problem 
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Figure  6.17:  Heat  Exchanger  System  Performance 
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Based  on  Table  6.3  it  can  be  concluded  that  it  is  most  important  to  tightly  control 
the  cold  fluid  flow  rate  through  the  heat  exchanger  given  that  the  optimal  uncertain 
intervals  for  those  two  parameters  are  all  less  than  4%,  determined  by  dividing  the 
uncertain  interval  for  each  parameter  by  the  maximum  parameter  value.  It  is  also  clear 
that  the  spacing  between  the  tubes  is  more  important  in  ensuring  low  variation  in  the 
system’s  objectives  (maximum  heat  transfer  rate  and  minimum  heat  exchanger  length) 
than  is  the  diameter  of  the  shell  or  even  number  of  tubes  in  the  shell  and  tube  heat 
exchanger.  These  results  not  exactly  intuitive  and  could  be  used  by  a  designer  going 
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forward  in  the  process  of  developing  the  heat  exchanger  by  focusing  more  attention  and 
resources  on  pumps  (to  control  flow  rate)  and  interior  geometry  (to  control  tube  space). 

As  with  the  previous  example,  these  solutions  are  as  close  to  the  deterministic 
Pareto  optimal  performance  as  possible  given  the  level  of  uncertainty  associated  with 
each  solution.  Since  the  design  space  for  this  problem  cannot  be  visualized  to 
demonstrate  this  fact  the  worst  case  objective  function  values  for  of  the  RUID  optimal 
solutions  included  in  Table  6.3  have  been  plotted  over  the  deterministic  optimal  solutions 
in  the  objective  space  for  the  problem  in  Figure  6.17.  In  the  figure  the  four  representative 
solutions  provided  in  Table  6.3  are  called  out  with  grey  fill  for  emphasis.  The  RUID 
solutions  are  shown  with  square  markers,  while  the  anchor  points  are  shown  with  circles. 
As  expected,  the  RUID  solutions  are  all  reasonably  close  to  the  deterministic  Pareto 
frontier,  suggesting  that  even  in  the  worst  case  the  uncertain  solutions  found  using  the 
RUID  approach  will  provide  objective  performance  that  is  very  close  to  the  deterministic 
optimal  performance  for  the  system. 

6.4.4  Meta-Model  Accuracy 

In  order  to  demonstrate  the  accuracy  of  the  meta-model  assisted  inner  problem 
used  by  the  RUID  approach,  each  of  the  obtained  Pareto  optimal  solutions  for  the  first 
two  example  problems  were  verified  by  recalculating  the  Vf  and  Vg  values  for  each 
optimal  pu  and  solution  obtained  using  actual  function  calls  vice  the  kriging  meta¬ 
models.  The  results  obtained  were  then  compared  and  a  percent  error  value  was 
calculated  assuming  the  function  call  obtained  solution  was  the  true  solution.  The 
average  percent  error  values  (ju)  along  with  the  standard  deviation  (<r)  of  those  percent 
error  values  are  provided  in  Table  6.4,  which  clearly  show  that  the  meta-model  assisted 
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RUID  approach  is  capable  of  accurately  representing  the  objective  and  constraint 


functions  of  the  example  problems  presented  in  this  chapter. 

Table  6.4:  Meta-Model  Accuracy  Statistics 

Vf  Percent  Error  Vg  Percent  Error 
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6.5  SUMMARY  OF  RESEARCH  THRUST  4 

A  new  approach  for  optimal  design  under  uncertainty,  called  Reducible  Uncertain 
Interval  Design  (RUID),  has  been  presented.  This  approach  is  focused  on  determining  the 
appropriate  level  of  uncertainty  for  reducible  sources  of  system  uncertainty.  RUID  does 
not  require  statistical  quantification  of  uncertainty,  which  may  be  difficult  to  obtain  or 
unavailable,  and  relaxes  many  of  the  assumptions  required  by  other  current  approaches. 
RUID  places  minimizing  objective  function  variation  in  tension  with  the  cost  of 
producing  that  minimal  variation  in  a  bi-objective  optimization  framework.  Furthermore 
the  RUID  approach  forces  the  worst  case  objective  performance  of  an  uncertain  design, 
in  a  multi-objective  sense,  to  be  as  close  as  possible  to  the  deterministic  Pareto  optimal 
performance  of  the  system.  The  efficiency  of  this  new  approach  is  significantly 
improved  through  the  use  of  kriging  meta-models  within  the  inner  problem  of  the  RUID 
algorithm.  The  kriging  meta-models  used  are  built  using  data  obtained  early  in  the 
algorithm  and  continually  refined  throughout  the  algorithm  in  order  to  ensure  accuracy. 
This  process  makes  it  possible  to  conduct  optimization  under  uncertainty  for  only  a 
slightly  higher  computational  effort  than  is  required  to  complete  a  deterministic 
optimization  of  the  system.  The  capabilities  and  efficiency  of  the  of  the  RUID  approach 
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were  demonstrated  through  three  example  problems  of  varying  difficulty.  These  example 
problems  showed  how  the  RUID  approach  can  be  used  to  determine  optimal  uncertain 
intervals  for  reducible  input  parameters  that  will  ensure  as  close  to  deterministic  Pareto 
optimal  performance  as  possible.  The  results  produced  by  the  RUID  approach  can  then 
be  used  by  designers  to  specify  tolerances,  materials  and/or  manufacturing  processes. 
Additionally  the  RUID  results  can  also  be  used  to  gain  a  better  insight  into  a  system’s 
uncertain  parameter  and  then  focus  attention  and  resources  on  those  uncertainties  that  are 
the  most  important  to  control  in  order  to  ensure  minimal  variations  in  the  system’s  overall 
performance. 

The  RUID  approach  makes  it  possible  for  designer  to  consider  reducible,  or 
controllable,  uncertainty  in  an  optimization  framework  in  an  efficient  manner  without  the 
need  for  potentially  erroneous  or  degrading  a  priori  assumptions  about  system 
uncertainty  levels.  This  capability  makes  it  possible  for  uncertainty  to  be  considered 
earlier  in  the  design  process  and  with  less  available  parameter  information.  The  results 
produced  by  this  new  approach  are  compelling,  especially  given  the  fact  that  the 
approach  requires  virtually  no  parameter  uncertainty  information.  However,  this  new 
approach  is  not  without  its  own  limitations  and  weaknesses.  First  and  foremost  this 
approach  requires  an  overall  increase  in  the  dimensionality  of  the  problem  as  it 
determines  upper  and  lower  bounds  on  all  parameter  values.  The  approach  also  requires 
that  two  optimization  procedures  be  performed  in  sequence  with  the  second  stage 
optimization  containing  a  set  of  nested  inner  optimization  problems.  Both  the  increase  in 
dimensionality  and  the  nested  structure  have  the  potential  to  cause  extreme  computational 
issues  for  many  problems,  including  the  second  example  presented  in  this  chapter. 
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Kriging  meta-models  were  used  in  the  approach  in  an  effort  to  reduce  this  computational 
burden,  but  meta-modeling  in  general  is  difficult  to  employ  effectively  for  some  systems 
and  possesses  its  own  drawbacks  including  decreased  accuracy.  Obviously  these  issues 
must  be  considered  before  employing  the  RUID  approach. 

However,  in  spite  of  these  drawbacks  the  new  RUID  approach  is  clearly  a  useful 
new  method  for  optimization  under  reducible  uncertainty  that  does  not  require  a  designer 
to  supply  numerous  pieces  of  potentially  unavailable  information  about  the  nature  and 
quantification  of  a  system’s  uncertain  parameters.  The  RUID  approach  is  capable  of 
producing  optimal  solutions  for  a  system  that  a  designer  can  then  use  to  specify  materials, 
manufacturing  process  and/or  tolerances.  The  solutions  produced  by  this  approach,  in  the 
form  of  upper  and  lower  bounds  on  all  uncertain  system  parameters,  will  assure  a 
designer  of  minimal  variations  in  a  system’s  objectives  for  a  minimal  cost  and  provide  as 
close  to  deterministic  optimal  performance  as  possible. 
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CHAPTER  7:  CONCLUSIONS 

This  dissertation  has  presented  four  distinct  strategies  for  understanding  and 
optimally  mitigating  the  degrading  effects  of  reducible  input  uncertainty  on  both  single 
and  multi-disciplinary  engineering  systems.  Chapter  1  discussed  the  motivation  for  this 
research,  along  with  the  specific  focus  and  assumption  of  this  work.  Chapter  2  then 
established  some  necessary  background  information  and  terminology  that  was  used 
through  the  dissertation.  Chapters  3  through  6  outlined  the  details  of  the  four  interrelated 
research  thrusts  of  this  dissertation.  Each  research  thrust  built  on  or  extended  the  work 
presented  in  the  preceding  thrust  in  an  attempt  to  overcome  inherent  limitations  or  to  add 
increased  functionality.  This  final  chapter  is  organized  as  follows:  Section  7.1 
summarizes  the  key  conclusions  of  each  research  thrust,  Section  7.2  details  the  main 
contributions  of  this  work  and  Section  7.4  provides  some  directions  for  future  research. 

7.1  DISSERTATION  SUMMARY 

In  this  section  each  of  the  four  research  thrusts  will  be  summarized  and  the  key 
conclusions  of  each  research  thrust  will  be  reviewed. 

7.1.1  Research  Thrust  1  Conclusions 

In  Chapter  3  a  new  global  sensitivity  analysis  approach  for  fully  coupled  multi¬ 
disciplinary,  multi-output  engineering  systems  under  uncertainty,  called  Multi¬ 
disciplinary  Multi-Output  Sensitivity  Analysis  (MIMOS)  was  presented.  This  approach 
extended  the  work  of  Li  et  al.  [2009a]  to  multi-disciplinary  systems  and  was  built  on  the 
multi-disciplinary  design  optimization  framework  proposed  by  Aute  and  Azarm  [2006]. 
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This  new  approach  assumes  that  input  parameter  uncertainty  is  quantified  in  the  form  of 
intervals  about  a  set  of  predetermined  nominal  parameter  values  for  a  candidate  design 
and  that  the  interval  uncertainty  is  reducible.  The  approach  then  solves  for  the  optimal 
level  of  uncertainty  reduction  at  both  the  system  and  sub-system  level  in  a  multi¬ 
objective  sense.  The  competing  objectives  in  this  approach  are  the  level  of  reduced 
variations  in  the  system  and  sub-system  outputs,  here  called  R ,  placed  in  tension  with  the 
cost  of  achieving  those  reduced  variations.  This  “cost”  is  a  function  of  the  required 
reduction  in  input  uncertainty  and  is  called  Investment  in  this  work. 

The  capabilities  of  the  MIMOSA  approach  were  demonstrated  through  two 
example  problems,  including  a  numerical  example  and  an  engineering  example. 
MIMOSA  was  shown  to  be  able  to  identify  both  critical  parameters  (the  uncertain  input 
parameters  that  have  the  greatest  effect  on  system  variations)  and  critical  sub-systems 
(the  sub-system(s)  that  have  the  greatest  effect  on  the  overall  performance  of  the  system 
as  a  whole  under  uncertainty).  Both  examples  specifically  present  how  the  MIMOSA 
approach  can  identify  the  sub-system  in  a  decomposed  multiple  sub-system  problem  that 
is  the  most  sensitive  to  the  uncertainty  and  has  the  greatest  effect  on  the  system  level 
performance  under  uncertainty.  The  engineering  example  also  demonstrates  how 
MIMOSA  is  capable  of  isolating  parameters  whose  uncertainty  greatly  influence  system 
and  sub-system  outputs  in  contrast  to  those  who  do  not. 

The  MIMOSA  approach  is  very  effective  but  not  without  its  downsides.  The 
approach  does  not  consider  engineering  feasibility  and  thus  in  not  effective  in  analyzing 
designs  where  input  uncertainty  levels  may  lead  to  potential  engineering  failure. 
Additionally  MIMOSA  requires  that  a  designer  possess  a  preexisting  or  preferred 
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candidate  design  in  the  form  of  nominal  parameter  values  and  known  parameter 
uncertainties,  which  may  not  always  be  available.  Furthermore,  the  MIMOSA  approach 
nests  a  sensitivity  analysis  optimization  problem  within  a  multi-level  MDO  framework, 
resulting  in  a  potentially  high  computational  cost  for  many  applications. 

7.1.2  Research  Thrust  2  Conclusions 

The  work  presented  in  Chapter  4  simply  extended  the  MIMOSA  approach  to  also 
consider  engineering  feasibility,  thus  effectively  overcoming  one  of  MIMOSA’s  major 
shortcomings.  This  new  sensitivity  analysis  approach  for  multidisciplinary  systems 
under  reducible  interval  input  uncertainty  is  called  Multi-dlciplinary  multi-objective 
Combined  Sensitivity  Analysis  (MICOSA).  In  contrast  to  MIMOSA  this  approach 
differentiates  between  the  objective  function  variations  and  constraint  function  variations 
for  a  candidate  design.  MICOSA  then  uses  sensitivity  analysis  techniques  to  determine 
the  optimal  combination  of  uncertainty  reductions  at  both  the  system  and  sub-system 
levels  that  will  optimally  reduce  the  variation  in  all  system  and  sub-system  outputs  while 
simultaneously  ensuring  engineering  feasibility. 

This  new  approach  was  demonstrated  through  both  a  single  disciplinary  example 
(in  the  form  of  a  UUV  propulsor  model)  and  a  multi-disciplinary  example  (the  entire 
UUV  system  in  a  decomposed  fashion).  The  results  obtained  were  shown  to  be  very 
similar  to  those  produced  by  the  MIMOSA  approach,  with  the  key  distinction  of  also 
being  able  to  guarantee  feasibility  under  any  retained  uncertainty.  The  single  disciplinary 
propulsor  model  example  specifically  demonstrated  the  capability  of  the  approach  to 
ensure  feasibility  through  a  side  by  side  comparison.  The  multi-disciplinary  UUV 
example  demonstrated  the  ability  of  the  MICOSA  approach  to  detect  critical  sub-systems. 
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Furthermore,  the  UUV  example  demonstrated  the  ability  of  the  Rc  metric  to  apply  limited 
uncertainty  reduction  resources  to  critical  constraint  variations  prior  to  using  uncertainty 
reduction  to  reduce  objective  variations. 

This  new  approach  is  a  significant  improvement  over  MIMOSA,  but  was  very 
limited  in  its  scope.  MICOSA  requires  that  uncertainty  reduction  alone  will  be  sufficient 
to  ensure  engineering  feasibility  under  retained  uncertainty  in  all  cases,  which  may  not 
necessarily  always  be  possible.  For  many  designs  no  amount  of  uncertainty  reduction 
may  be  sufficient  to  ensure  feasibility.  Additionally  MICOSA  also  shares  MIMOSA 
potential  for  high  computational  cost  due  to  the  nested  structure  of  the  approach. 

7.1.3  Research  Thrust  3  Conclusions 

To  overcome  the  key  limitation  of  the  MICOSA  approach,  its  assumption  that 
uncertainty  reduction  alone  is  sufficient  to  ensure  feasibility  under  uncertainty,  a  new 
sensitivity  analysis  approach  for  single  disciplinary  systems  was  developed.  This 
approach,  called  Design  Improvement  by  Sensitivity  Analysis  (DISA)  was  again  based 
on  the  work  of  Li  et  al.,  but  it  not  limited  to  uncertainty  reduction  alone.  Like  MIMOSA 
and  MICOSA  it  places  reduced  system  output  variations  in  tension  with  the  cost  to 
achieve  those  reduced  variation  in  a  multi-objective  sense,  but  allows  not  just  for 
uncertainty  reduction  but  also  for  small  changes  to  the  nominal  design  of  the  system. 
Those  small  changes,  called  design  adjustments,  are  the  key  to  being  able  to  ensure 
feasibility  under  uncertainty.  The  DISA  approach  is  performed  in  two  distinct  stages. 
During  the  first  stage  the  system  is  analyzed  with  respect  to  uncertainty  reduction 
opportunities.  Then  in  the  second  stage  the  nominal  parameter  values  of  the  candidate 
design  are  adjusted  as  necessary  (given  some  level  of  uncertainty  reduction  as  determined 
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in  the  first  stage)  in  an  effort  to  produce  design  solutions  that  possess  an  optimal  level  of 
uncertainty  reduction  but  are  also  guaranteed  to  be  feasible.  The  two  stage  structure  of 
the  DISA  approach  also  makes  it  possible  to  use  system  analysis  information  generated 
during  the  first  stage  to  build  surrogate  models  of  the  system  for  use  in  the  second  stage 
procedure,  thus  greatly  increasing  the  computational  efficiency  of  the  approach. 

The  capabilities  of  the  DISA  approach  were  demonstrated  through  two  example 
problems  of  varying  complexity.  The  DISA  approach  was  applied  to  a  simple  two- 
dimensional  example  problem  and  a  more  complex  engineering  example.  The  simple 
example  clearly  demonstrated  the  ability  of  the  DISA  approach  to  use  small  design 
adjustments  to  change  the  nominal  parameter  values  of  a  candidate  design  in  order  to 
move  the  candidate  design  away  from  active  constraints  and  ensure  the  feasibility  of  the 
design  under  uncertainty.  With  the  complex  example  a  specific  study  was  performed  on 
a  single  candidate  solution  produced  by  the  DISA  approach  to  demonstrate  the  ability  of 
the  new  method  for  showing  a  designer  which  specifics  elements  of  an  engineering 
system  are  the  most  critical  to  both  the  feasibility  and  performance  of  the  design.  More 
importantly,  it  was  also  shown  through  both  examples  that  the  DISA  approach  is  also 
capable  of  suggesting  to  a  designer  how  best  to  use  available  investment  through 
uncertainty  reduction  and/or  small  design  adjustments. 

The  approach  proved  to  be  very  capable  at  both  ensuring  feasibility  and  reducing 
system  variations,  in  some  cases  to  an  even  greater  extent  than  would  be  possible  through 
uncertainty  reduction  alone.  However,  the  DISA  approach  like  the  MIMOSA  and 
MICOSA  approaches  required  a  large  amount  of  known  or  predetermined  system 
information  in  the  form  of  a  candidate  design  and  known  parameter  uncertainty  intervals 
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about  the  nominal  parameter  values  of  the  candidate  design.  These  a  priori  requirements 
are  limiting  the  cases  where  a  designer  is  faced  with  sources  of  reducible  system 
uncertainty,  but  does  not  yet  have  a  system  design  in  place  to  analyze.  Furthermore,  the 
DISA  approach  relies  on  surrogate  approximation  techniques  (specifically  kriging)  to 
ensure  a  reasonable  computational  effort.  Surrogate  approximation  techniques  are  not 
without  their  own  specific  sets  of  drawbacks  and  limitations  [Shan  and  Wang,  2008]. 
Additionally,  the  DISA  approach  is  currently  only  applicable  to  single  disciplinary 
systems. 

7.1.4  Research  Thrust  4  Conclusions 

As  a  result  of  the  limiting  requirements  of  the  previous  three  research  thrusts,  a  new 
more  general  approach  for  the  design  of  multi-objective  engineering  systems  under 
reducible  interval  uncertainty  was  developed.  This  approach,  called  Reducible  Uncertain 
Interval  Design  (RUID)  eliminates  the  need  for  candidate  designs,  known  uncertain 
intervals  and  nominal  parameter  values.  RUID  simple  searches  the  parameter  space  of  a 
system  in  an  effort  to  find  the  optimal  upper  and  lower  bounds  for  all  system  input 
parameters  that  are  known  to  possess  reducible  uncertainty.  The  solutions  produced  are 
optimal  in  the  sense  that  RUID  places  minimizing  objective  function  variation  in  tension 
with  the  cost  of  producing  that  minimal  variation  in  a  bi-objective  optimization 
framework.  Furthermore  the  RUID  approach  forces  the  worst  case  objective 
performance  of  an  uncertain  design,  in  a  multi-objective  sense,  to  be  as  close  as  possible 
to  the  deterministic  Pareto  optimal  performance  of  the  system.  The  computational 
efficiency  of  the  approach  is  again  improved  through  the  use  of  surrogate  models  which 
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are  again  developed  using  information  obtained  early  in  the  approach  and  then  refined 
over  time  as  the  approach  converges  to  the  optimal  uncertain  interval  solutions. 

The  capabilities  of  the  RUID  approach  were  demonstrated  through  three  example 
problems  of  varying  complexity  and  the  results  produced  were  quite  compelling.  The 
first  two  example  problems  presented  where  both  two-dimensional  problems  of  varying 
complexity  and  both  graphically  demonstrated  the  capability  of  the  approach  to  produce 
optimal  upper  and  lower  bounds  for  all  reducible  input  parameter  that  guaranteed 
feasibility  while  simultaneously  being  as  close  to  the  deterministic  Pareto  frontier  of  the 
problem  as  possible.  The  complex  engineering  example  further  demonstrated  the  ability 
of  the  approach  to  draw  a  designer’s  attention  to  an  engineering  system’s  most  critical 
uncertain  input  parameters. 

The  RUID  approach  is  a  sensitivity  analysis  inspired  approach  for  the  design  of 
multi-objective  systems  under  reducible  systems  that  does  not  require  statistical 
information  or  any  other  limiting  a  priori  information  about  the  system  or  its  inputs.  This 
makes  the  RUID  approach  useful  to  designers  focused  on  answering  uncertainty 
reduction  questions  much  earlier  in  the  design  process  and  with  much  less  required 
information  and/or  system  knowledge.  The  main  drawback  of  the  RUID  approach  is  the 
associated  computational  effort.  Attempts  were  made  to  address  this  issue  through  the 
use  of  kriging  meta-models,  but  as  pointed  out  in  the  previous  subsection  meta-modeling 
is  not  without  its  own  significant  drawbacks.  As  with  all  the  approaches  presented  in  this 
dissertation,  virtually  all  approach  for  design  under  uncertainty  must  determine  the 
effects  of  uncertainty  on  any  design  solution  considered  within  the  approach.  This 
assessment  of  the  effects  of  uncertainty  often  involves  an  optimization  procedure,  which 
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necessarily  then  requires  that  the  approach  consists  of  solving  nested  optimization 
problems.  This  nested  structure  is  what  produces  the  high  computational  expense  of  the 
RUID  approach,  and  many  other  approaches  optimization  under  uncertainty  approach  for 
that  matter.  Furthermore,  RUID  is  only  presently  applicable  to  single  disciplinary 
systems. 

7.2  MAIN  CONTRIBUTIONS 

Several  new  and  novel  sensitivity  analysis  based  approaches  for  optimally 
mitigating  the  effects  of  reducible  interval  uncertainty  on  single-  and  multi-disciplinary 
systems  have  been  presented  in  this  dissertation.  The  specific  key  contributions  of  this 
work  to  the  design  research  community  are  outlined  in  the  following  paragraphs. 

The  MIMOSA  approach  presented  in  Chapter  3  provided  a  new  framework  for 
analyzing  the  effect  of  reducible  interval  uncertainty  on  fully  coupled  multi-disciplinary, 
multi-output  systems.  As  previously  stated  this  approach  is  capable  of  determining  the 
relative  importance  of  different  uncertain  inputs  and  of  different  sub-system  with  regards 
to  their  impact  on  the  overall  performance  of  the  system  as  a  whole.  Furthermore  the 
MIMOSA  approach  provides  designer  with  the  combination  of  uncertainty  reduction 
required  to  optimally  mitigate  the  effect  of  uncertainty  on  the  system  and  sub-system 
designs.  The  MISMOSA  approach  was  presented  in  part  in  Li  et  al.  [2009b]. 

The  DISA  approach  presented  in  Chapter  5  provides  a  sensitivity  analysis  based 
approach  for  optimally  mitigating  the  effects  of  reducible  input  uncertainty  on  a 
preexisting  candidate  design  that  is  capable  of  both  ensuring  engineering  feasibility  and 
also  providing  minimal  objective  function  variations  at  a  minimal  require  cost  through 
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the  use  of  uncertainty  reduction  mechanisms  and  small  design  adjustments.  The  DISA 
approach  is  capable  of  providing  designers  with  solutions  that  suggest  optimal 
combinations  of  uncertainty  reduction  and  small  nominal  parameter  value  adjustments 
that  will  improve  the  overall  performance  of  a  candidate  design  with  respect  to  both 
objective  function  variations  and  engineering  feasibility  under  uncertainty.  The  DISA 
approach  was  presented  in  part  in  Hamel  et  al.  [2010]. 

The  RUID  approach  presented  in  Chapter  6  provides  new  multi-objective  design 
under  uncertainty  approach,  inspired  by  the  previously  developed  SA  approaches,  that 
can  produce  optimal  solutions  in  the  form  of  upper  and  lower  bounds  (which  specify 
uncertain  intervals)  for  all  input  parameters  to  a  system  that  possess  reducible 
uncertainty.  The  solutions  produced  by  this  approach  provide  minimal  variation  in  system 
objectives  for  a  maximum  allowed  level  of  input  uncertainty  in  a  multi-objective  sense 
and  furthermore  guarantee  as  close  to  deterministic  Pareto  optimal  performance  as 
possible  with  respect  to  the  uncertain  parameters.  This  approach  requires  a  very  limited 
amount  of  a  priori  information  and/or  assumptions  and  thus  makes  it  much  easier  for 
designers  to  consider  the  degrading  effects  of  reducible  uncertainty  much  earlier  in  the 
design  process.  The  RUID  approach  was  presented  in  part  in  Hamel  and  Azarm  [2010]. 

7.3  FUTURE  RESEARCH  DIRECTIONS 

The  final  section  of  the  dissertation  outlines  some  directions  for  potential  future 
research.  These  ideas  either  build  on  or  extend  the  work  presented  in  this  dissertation  or 
seek  to  overcome  some  of  the  shortcoming  of  the  work  presented. 
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7.3.1  Multi-Disciplinary  Extension  of  the  DISA  and  RUID  Approaches 

A  clear  next  step,  as  alluded  to  earlier  in  this  chapter,  would  be  to  develop  multi¬ 
disciplinary  extensions  of  the  approaches  presented  in  Chapter  5  and  6.  These  two  new 
approaches  are  very  capable  and  have  many  compelling  attributes,  but  are  thus  far  only 
applicable  to  single  disciplinary  systems.  This  fact  limits  their  applicability  to  real-world 
engineering  design  challenges,  which  as  discussed  in  Chapter  1  will  often  be  multi¬ 
disciplinary  in  nature.  The  framework  developed  for  the  MIMOSA  approach  detailed  in 
Chapter  3  could  provide  a  good  starting  point  for  this  work  and  multi-disciplinary 
extension  of  both  the  DISA  and  RUID  approaches  would  be  particularly  useful. 

7.3.2  Approaches  for  Mixed  Reducible  and  Irreducible  Uncertainties 

All  the  work  presented  in  this  dissertation  has  focused  on  reducible  uncertainty, 
which  is  clearly  an  important  aspect  of  engineering  design.  However,  irreducible 
uncertainty  is  also  obviously  a  critical  issue  as  evidenced  by  the  numerous  robust 
approaches  that  appear  throughout  the  literature  (see  [Beyer  and  Senhoff,  2007]).  As 
discussed  earlier  in  this  thesis  a  few  recent  approaches  have  attempted  to  combine 
irreducible  sources  of  uncertainty  and  uncertainty  reduction  mechanisms  in  a  few 
different  ways  (e.g.  [Qu  et  al.,  2003;  Li  et  al.,  2009c,  Wang  et  al.,  2009]).  This  area  of 
research  deserves  more  attention  and  would  be  useful  to  real-world  design  efforts.  The 
distinction  between  reducible  and  irreducible  uncertainties  is  somewhat  subjective  and 
thus  distinguishing  between  these  two  ways  of  looking  at  uncertainty  is  limiting.  It  is 
extremely  likely  that  a  designer  may  not  be  able  to  classify  various  sources  of  uncertainty 
as  either  purely  reducible  or  strictly  irreducible,  especially  in  the  early  stages  of  a  design 
process.  Careful  work  should  be  undertaken  to  develop  new  approaches  that  can  reliably 
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consider  both  reducible  and  irreducible  sources  of  uncertainty  and  that  allow  designer 
with  freedom  to  change  the  way  uncertainty  is  classified  should  better  information 
become  available. 

7.3.3  More  Efficient  Nested  Optimization  Algorithms 

Chapter  1  clearly  motivates  the  need  for  more  efficient  strategies  for  solving  nested 
optimization  problems.  Most  current  SA  and  design  under  uncertainty  approaches  rely 
heavily  on  multi-disciplinary  optimization  and  decomposition  techniques.  As  previously 
discussed,  there  are  numerous  MDO  approaches  currently  reported  in  the  literature,  but 
the  work  that  has  been  done  up  to  this  point  in  the  area  has  proved  to  be  relatively 
inefficient  from  a  computational  effort  standpoint.  Current  decomposition  techniques, 
such  as  Collaborative  Optimization  and  Analytical  Target  Cascading,  involve  nesting 
sub-system  design  algorithms  inside  a  coordinating  or  system-level  design  algorithm  at  a 
high  computational  expense  [Balling  and  Sobieszczanski-Sobieski,  1996;  Aute  and 
Azarm,  2006;  Kokkolaras  et  al.,  2006;  Yi  et  al.,  2007].  As  seen  throughout  this 
dissertation,  nested  optimization  techniques  are  frequently  used  when  solving  sensitivity 
analysis  problems  and  the  work  presented  in  this  dissertation  was  repeatedly  impacted  by 
the  high  computational  effort  of  those  techniques.  Clearly  more  work  is  needed  in  this 
area  before  SA  approaches  and  become  more  efficient  and  useful  to  practicing  design 
engineers.  Thus  far  the  only  real  answer  to  these  efficiency  issues  have  been  to  use  meta¬ 
modeling  techniques  to  approximate  expensive  analysis  models  (such  as  CFD  codes) 
with  less  expensive  analytical  functions  [Martin  and  Simpson,  2005;  Wang  and  Shan, 
2007;  Shan  and  Wang,  2008].  Meta-models  can  be  used  to  reduce  the  underlying 
computational  expense  associated  with  the  analysis  functions  of  a  design  system  quite 
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effectively,  as  demonstrated  in  Chapters  5  and  6,  but  their  use  does  nothing  to  lessen  the 
efficiency  issues  associated  with  the  nested  algorithms  currently  used  to  solve 
multidisciplinary  problems.  Clearly  other  means  for  improving  the  efficiency  of  multi¬ 
disciplinary  algorithms  are  needed,  particularly  for  problems  that  already  use  meta¬ 
modeling  but  are  still  too  computationally  expensive  to  solve  efficiently.  Perhaps  new 
applications  and/or  extensions  of  other  classical  decomposition  techniques,  such  as 
Bender’s  Decomposition  [e.g.  Conejo  et  al.,  2006],  could  provide  the  answer  to  this  very 
real  and  challenging  problem  that  must  be  addressed  in  the  near  future. 
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